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Summary 


A dynamic real-time simulation study was conducted at the NASA Langley Research Center 
to gather comparative performance data among three candidate final-approach spacing aid (FASA) 
display formats. The study was funded jointly by NASA and FAA and defined in collaboration 
with Lincoln Laboratory, MIT, and NASA Ames Research Center. Prior to formal publication, the 
experimental results were presented to the FAA for use in their Terminal ATC Automation (TATCA) 
Program early field implementation decisions to define the final controller’s automation aid interface. 
Several objective measures of controller performance and their eye-scan behavior, together with 
subjective workload and rating questionnaires w T ere used to obtain an in-depth assessment . The data 
were gathered by using 12 subject controllers provided by the FAA. For each of two representative 
pattern-speed procedures (a 170-knot procedure and a 210-knot procedure with speed control aiding), 
data were collected by using 4 final-controller display-format conditions: manual/ARTS III, graphic 
marker, DICE countdown, and centerline slot marker. In addition to the experimental results and 
a simple runway- arrival- rate analysis, key FASA issues, a rationale for selecting the tested formats 
and their description, are presented. 

Based on objective measures, the graphic marker and DICE countdown format were both more 
precise than the centerline slot marker in terms of delivery time errors at the runway threshold, 
and both (graphic and DICE) improved delivery precision relative to the manual/ARTS III format. 
In addition to having the least delivery precision, of the three FASA formats tested, the centerline 
slot marker had the least amount of controller monitoring of aircraft inside the final-approach fix. 
Although relatively close to each other in delivery performance, eye-scanning analysis indicated 
the graphic marker appears to have a quicker or more efficient information transfer process than the 
DICE countdown format. The 210-knot pattern-speed procedure formats provided better interarrival 
precision relative to the corresponding 170-knot procedure formats. This may indicate the potential 
benefit of automation providing speed control advisories after the base-to-final turn where a higher 
pattern speed is practical in that region. Depending on which pattern-speed procedures are assumed 
in a simple runway-arrival-rate analysis, the improved precision of a FASA, such as the graphic 
marker, has the potential to increase the TATCA IMC (instrument meteorological conditions) arrival 
rate somewhere between 6 to 16 percent over that of a TATCA system without a FASA. All FASA 
formats reduced the number of vectors issued by our pool of final controllers, relative to their 
manual/ARTS III format; however, the graphic marker and DICE countdown reductions were 1.6 
to 2.1 times that of the centerline slot marker. 

Based on Task Load Index (TLX, a subjective workload assessment technique) evaluation, for 
the 170-knot pattern-speed procedure, the centerline slot marker increased workload above the 
manual/ARTS III format, whereas the graphic marker reduced workload relative to the same manual 
baseline. The DICE TLX-rated workload fell between that of the manual and the graphic formats. 
On the other hand, as a group, formats for the 2 10- knot procedure had no TLX workload difference 
among its formats. Also as a group, the 210-knot procedure formats did not have higher TLX 
workloads than the 170-knot procedure formats as a group, even though the 210-knot delivery 
precisions were significantly better. Each of three format, relative rating questionnaires (formats of 
only the 170-knot procedure, formats of only the 210-knot procedure, and all the formats of both the 
170-knot and 210-knot procedures) was designed to extract from the subjects their relative ranking 
of the formats with respect to three specific criteria (workload or effort required to use the format, 
ease of adapting to or learning to use the format, and amount of help or benefit in spacing traffic on 
final). In all cases, the following same FASA order of preference resulted: graphic marker, DICE, 
and slot marker. Additionally, when rating all the formats of both the 170- and 210-knot procedures 
together, in every format case, the 210-knot format was preferred over the corresponding 170-knot 
format. In their final debriefing, the subjects were unanimous in their feeling that automated aids 
would be beneficial in reducing workload and in increasing spacing precision. 
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1. Introduction 

The FAA has established the Terminal ATC Au- 
tomation (TATCA) Program to develop and test 
ATC automation aids for assisting controllers in the 
organization and control of arriving aircraft traffic 
in the extended terminal area (i.e., a major termi- 
nal and its enveloping en route airspace). The ex- 
pected benefits are more efficient utilization of run- 
ways and reduced controller workload. The functions 
to be addressed by TATCA include 

Traffic planning aid 
Descent advisory 
Final spacing aid 
Converging runway display aid 

Currently the underlying structure for the first three 
functions above is the Center/TRACON Automation 
System (CTAS) tools developed at the NASA Ames 
Research Center (refs. 1 to 3). The fourth function 
is described in reference 4. 

This report describes a study conducted at the 
NASA Langley Research Center to expand the 
TATCA knowledge base. NASA and FAA shared 
the experimental cost and Lincoln Laboratory, MIT, 
provided inputs to the experimental design and 
controller subject questionnaires. Working jointly 
with the FAA, Lincoln Laboratory", and Ames, a 
Langley study was conducted to help identify the 
most promising final- approach spacing aid (FASA) 
format for use in the TATCA early field implemen- 
tation. To that end, the collected data were quickly 
shared with these parties prior to formal publication. 

The TRACON environment at early TATCA im- 
plementation is expected to still use monochrome 
ARTS (automated radar terminal systems) displays 
before the introduction of the Advanced Automa- 
tion System (A AS) controller suite color displays. 
The study was directed toward gathering final- 
controller comparative-performance data, among po- 
tential FASA formats using a monochrome (no color) 
display. The final controller’s primary responsibil- 
ity is the acceptance of traffic from the arrival po- 
sition and merging and spacing traffic for the final 
approach. 

Separation and delivery precision, controller eye- 
scan of the radar display, number of vectors (head- 
ing changes) issued, response time, workload, and 
questionnaire data were gathered. These multiple 
measures provided a broad assessment of the rela- 
tive performance of the formats and, in some cases, 
resolved ambiguity. The data were collected from 
12 subject controllers provided by the FAA, all of 


whom were active, full performance level (FPL) ter- 
minal area controllers. The subjects served as the 
final controller in a real-time TRACON (terminal 
radar approach control) simulation under several ex- 
perimental conditions. For each of two representative 
approach pattern-speed procedures, the controllers 
ran a data session in a manual mode (no automa- 
tion spacing aid) followed by data sessions employing 
three final-approach spacing aid (FASA) formats, 

2. Background 

The late 1950’s, the 1960’s, and the early 1970’s 
witnessed considerable activity in the area of 
computer-aided spacing systems for terminal area 
ATC. Reference 5 contains an excellent summary and 
bibliography of that activity. Reference 6 lists some 
of the reasons why an operational computer-aided 
spacing system was not accepted. Among the rea- 
sons were the limitation of computer, display, and 
tracking technology at that time. The automation 
aid interface to the controller is a key component in 
an acceptable computer-aided spacing system for the 
terminal area. 

Air travel delay and traffic congestion at major 
airports, projected increases in air travel, and severe 
environmental restrictions on either airport expan- 
sion or new airport construction all signal the press- 
ing need for maximum utilization of present airport 
real estate. These conditions have stimulated a new 
effort to develop and test computer-aided, time-based 
air traffic control systems for the extended terminal 
area. References 1, 2, 3, and 6 through 15 document 
some of the more recent developments and tests in 
the area. In the United States, these events have 
stimulated the formation of the FAA Terminal ATC 
Automation (TATCA) Program briefly described in 
the introduction of this report. 

For the near term, a computer-aided system is 
believed to have the potential for more consistent 
spacing, at current operational separations, with less 
controller workload than can be achieved with the 
present system. Even more noteworthy are the po- 
tential long-term benefits of a computer-aided, time- 
based air traffic control system for the extended ter- 
minal area. Final- approach longitudinal separation 
from the preceding aircraft is mandated by two time- 
based constraints: the time required for the wake 
vortex of the preceding aircraft to decay to a safe 
encounter level and its runway occupancy time. Re- 
search and developments in the area of improved air- 
craft flare and touchdown, runway guidance, high- 
speed turn-offs, accurate weather prediction, and 
wake vortex modeling and detection could eventually 



permit the use of variable time separations between 
approaching aircraft. These variable time separa- 
tions would be updated, as a function of conditions, 
to the minimum times required to satisfy the two 
previous time-based separation constraints. There 
are additional potential benefits to a time-based sys- 
tem such as allowing aircraft to employ their onboard 
four-dimensional (4-D) flight management systems to 
precisely meet their desired metering fix times in a 
fuel efficient manner. Subject to constraints, this ap- 
proach might later be extended to allow aircraft to 
meet their ground-issued scheduled landing times. 

3, FASA Issues and Format 
Description 

3.1* FASA and Approach Speed Issues 

The ATC system characteristics assumed are de- 
scribed in this section. FASA issues, classification of 
t} r pes, and choices of FASA formats studied are dis- 
cussed. The relation of FASA delivery precision and 
pattern-speed procedure is examined. 

3.1.1. ATC System Environment 

Description 

As mentioned in the introduction, the purpose 
of the FAA TATCA program is to develop and test 
ATC automation tools for assisting in the organi- 
zation and control of arrival traffic in the extended 
terminal area. This area includes the surrounding 
en route traffic flow management to the terminal of 
interest (en route cruise and descent control), in ad- 
dition to the TRACON’s own feeder and final sec- 
tor traffic control. This study looked at the bene- 
fits of automation assistance to the final controller 
with the assumption that the functional elements of 
the Centcr/TRACON Automation System (CTAS) 
(refs. 1 and 2) were operating. That is, the Traf- 
fic Management Advisor (TMA) assisted the Center 
and TRACON traffic managers with sequencing and 
scheduling the traffic, the Descent Advisor assisted 
Center controllers in meeting the TMA’s schedule 
safely, and the Final- Approach Spacing Tool (FAST) 
assisted the TRACON feeder controllers in updat- 
ing the initial sequence and fine-tuning the traffic 
flow. The FASA is the technique used to display, to 
the final controller, the spacing, sequence, and ATC 
actions suggested by the FAST. 

A feature modeled in the simulation was that the 
algorithms (FAST) driving the FASA’s had knowl- 
edge of the pilot’s planned final- approach speed. The 
factors which primarily influence the pilot’s choice 


of final-approach speed are wind and aircraft land- 
ing weights. It is expected that later generations of 
FAST will obtain the pilot’s planned final-approach 
speed via data link in order to more precisely sched- 
ule and space aircraft. Final-speed knowledge was as- 
sumed because differences of controller performance, 
resulting from the FASA formats evaluated, can be 
experimentally differentiated more clearly under that 
higher capability condition. Unknown landing speeds 
add considerable uncertainty to landing time calcu- 
lation and thus would reduce the ability to discrim- 
inate any differences in delivery performance due to 
the FASA formats tested. 

The display environment simulated was that of 
an advanced ARTS monochrome display prior to the 
TRACON acquisition of the AAS controller suite 
plan position indicators (PPI’s) with color displays. 
The FAST system advisories are expected to be 
placed on the full digital ARTS display (FDAD) 
briefly described in reference 16. This precluded 
using the discrimination power of color coding to 
display FASA information. In addition, the added 
FASA information could not displace or significantly 
detract from the primary ARTS information. 

3.1.2. Final- Approach Control and 

Automation Issue 

Given the ATC environment described above, this 
study evaluated the performance of the final con- 
troller in directing aircraft to final approach with 
consistent and proper separation under the follow- 
ing conditions: (1) using only manual judgment/ 

experience (with ARTS III display) and (2) em- 
ploying the assistance of the various FASA’s. It is 
important to note that the manual/ A RTS format 
evaluated does not represent today’s TRACON envi- 
ronment where the traffic flow to the final controller 
is the end product of a scries of merges performed 
manually on different traffic streams by other con- 
trollers. The traffic in this study was assumed to have 
been organized and spaced by automat ion /controller 
interaction prior to arrival in the final controller’s 
airspace. Generally, gaps appeared when required in 
the arrival stream to merge traffic and the landing 
order was normally self-evident. The experimental 
manual/ ARTS format addressed the question of how 
precisely a controller can manually vector and sepa- 
rate traffic if given a well-organized stream of traf- 
fic, essentially an “ideal feed.” Performance of this 
study’s manual/ARTS format together with FASA 
performances addresses a bigger system issue. That 
issue is whether, and to what extent, a FASA is ben- 
eficial if the TATCA system (i.e., CTAS) has already 



been active in the organization and tentative spacing 
of arrival traffic prior to the final sector. 

3.1.3. Classification of FASA Types 

An attempt was made to classify the many ap- 
proaches possible to display, on the final controller’s 
PPI, computer assistance from a dynamic time-based 
planner (DTP) such as exists in CTAS. For the en- 
vironment depicted in the previous section, the key 
issue is selecting which technique best improves the 
final controller’s aircraft-pair spacing precision. Re- 
lated are issues of safety, stress, workload, accep- 
tance, and job satisfaction. A listing, under descrip- 
tive categories, of the viable techniques known to the 
authors is as follows: 

Category A — DTP aircraft schedules indication 

T Sequence list with STA’s (scheduled time of 
arrival): an aircraft/STA list ordered by STA 
values at the runway threshold 

2. Time line display with STA’s: a linear time 
scale with aircraft call signs positioned to in- 
dicate their STA’s relative to current time at 
the runway threshold (normally at bottom of 
time scale) 

Category B DTP conformance indication 

1. Sequence list with STA’s, early or late status: 
category Al together with aircraft expected 
arrival status information (early or late indi- 
cation) if nominal approach profile is followed 
(i.e., if no corrective action taken) 

2. Sequence list with STA’s and ETA’s (esti- 
mated time of arrival): category Al together 
with expected runway threshold time if nomi- 
nal approach profile is followed 

3. Sequence list with STA’s, numerical expected 

landing time errors: category Al together 

with expected aircraft landing time error 
(STA — ETA) if nominal approach profile is 
followed 

4. Time line display with STA’s, early or late 

status: category A 2 together with aircraft 

expected arrival status information (early or 
late indication) if nominal approach profile is 
followed 

5. Time line display with STA’s and ETA’s: cat- 
egory A2 together with aircraft call sign (nor- 
mally on the opposite side of the time scale 
from the STA call sign and usually colored 
or displayed differently) positioned to show 
their ETA’s relative to their STA’s if nominal 
approach profile is followed 


6. Time line display with STA’s, numerical ex- 
pected landing time errors: category A2 to- 
gether with expected aircraft landing time er- 
ror (STA — ETA) if nominal approach profile 
is followed 

7. Expected landing time error numerically dis- 
played in data block 

8. Extended runway centerline slot marker: in- 
dication of the desired (scheduled) position of 
arriving aircraft as if they were approaching 
along the extended runway centerline at the 
final pattern speed 

Category C — vector heading advisor 

1. Numerical heading: magnetic vector heading 
usually added to data block 

2. Graphical heading: symbolic indication on 

PPI (directed line, arrow, etc.) 

Category D — speed advisor 

1. Numerical speed: indicated airspeed usually 
added to data block 

Category E — advisory delivery point (position or 
time) indication 

1. Intensity (brightness) fluctuation of numerical 
data 

2. Intensity fluctuation of aircraft position 
symbol 

3. Graphical position indication: graphical sym- 
bol located on PPI 

4. Straight clock numerical countdown: time re- 
maining before issuing advisory 

5. DICE (direct course error) numerical count- 
down: amount of time relative to its STA 
(early or late), an aircraft would be if its 
advisory was issued immediately 

6. Symbolic clock face: pictorial representation 
of category E4 

7. Rising or falling time column: linear represen- 
tation of category E4 

Another characterization of the automation aids 
is obtained by classifying them as either indirect or 
direct. Those in category A or B are classified as 
indirect because the specific manner of achieving the 
desired schedule is not given. A combination of an 
aid from category E together with one from C or D is 
called direct , since the specific advisory to achieve the 
DTP schedule and the specific moment to deliver the 
advisory are explicitly indicated. In general, greater 
precision would be expected from direct aids. On the 
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other hand, the controller’s job satisfaction (making 
decisions, being in charge) might better be served by 
indirect aids. 

Indirect aids indicate the end goals; however, the 
specific control action necessary to meet that end and 
the point to apply it are left to controller judgment. 
Therefore feedback seems desirable to indicate the 
effectiveness of the control action taken. Category B 
has this desired feedback. 

Early or late information in categories B1 
through B7 is determined by comparing DTP sched- 
uled arrival time with the estimated arrival time 
which is a function of the time to fly to and then 
follow a DTP assumed nominal speed and path pro- 
file from the aircraft’s current speed and position in 
space. Because routes are generally better defined in 
the merge or feeder controller’s area, categories B1 
through B7 are better suited to the feeder than the fi- 
nal controller. For example, let us take an aircraft on 
the downwind leg with no FAS A. The aircraft could 
be subsequently given one to three vectors and one or 
two speed reductions. Several combinations of legit- 
imate controller actions could result in meeting the 
schedule. In that context, early or late is somewhat 
difficult to define if the aid is to remain indirect. Cat- 
egory B8 has the attribute of remaining indirect yet 
providing effective feedback. 

The British PACTAS (Predictive Approach Con- 
trol Tactical Advisor System) program (ref. 14) is a 
hybrid (indirect/direct) system that operates as cat- 
egory B7 or can operate as category E5 if the con- 
troller inputs a specific control action (heading or 
speed change). In its direct mode the PACTAS sys- 
tem provides DICE countdown information to the 
controller relative to the immediate initiation of the 
particular control action specified. This approach al- 
lows the controller to select the “how” (specific con- 
trol action) as normally done with an indirect aid 
and also provides the “when” (time) of a direct aid. 
However, the additional display and input devices 
required for PACTAS preclude its application in the 
environment described in section 3.1.1. 

3.1.4. FASA Formats Selected for Study 

The indirect aid selected for evaluation was the 
extended runway centerline slot marker (ref. 12). 
This format displays the spatial mapping of the se- 
quence and desired spacing of aircraft on the ex- 
tended centerline of the runway and is described in 
detail in section 3.2.2. During busy periods at ma- 
jor terminals, the final controller’s attention is highly 
focused on the aircraft positions and overall flow pat- 
tern on the PPI in order to plan and control the air- 


craft in the controller’s airspace. The actual position 
of aircraft as they turn on final as compared with 
the DTP desired positions is obtained simply and di- 
rectly by displaying the centerline slot markers in the 
region of normal controller eye scan. 

Even though it is possible to use some forms of 
direct aids with sequence lists or time line displays, 
as it relates to the final controller, we will use the 
term “direct” to apply only to aids displayed both 
on the PPI (not on an auxiliary display) and within 
the final controller’s normal scan pattern for traffic. 
The direct aids, as defined, can be broken into two 
additional categories. The first indicates the sug- 
gested aircraft location where specific ATC instruc- 
tions should be issued. The second category indicates 
the suggested time when specific ATC instructions 
should be issued. One direct FASA representative of 
each of these two categories was selected for evalu- 
ation. Other factors in narrowing the choices were 
the desire to give the controller some lead time rel- 
ative to the message delivery point and to minimize 
distracting clutter on the controller’s display. 

The location direct turning aid selected for eval- 
uation is a monochrome modification of the color- 
coded graphical advisory interface described in ref- 
erence 3. This reference used a two-segment symbol 
which conveys in a simple and direct graphical man- 
ner both the aircraft location to issue a turn in- 
struction and the desired heading. To prevent dis- 
play clutter, the graphical turn symbol in this study 
was changed to a three-segment symbol because the 
two-segment symbol quickly became elongated and 
unwieldy if the proposed change in heading was 
larger than 90°. The graphic format evaluated (cat- 
egories E3 and C2) is described in detail in sec- 
tion 3.2.4. It should be noted that the NASA Ames 
FAST simulation now uses an “x” and an arc symbol 
to indicate turns (ref. 15). 

Of the time direct aids listed, only the DICE 
countdown and the straight clock countdown possess 
both the features of lead time and relatively mini- 
mum display clutter. The desired heading and count- 
down value are both encoded on additional lines of 
the ARTS III data block. Prior to a turn, the DICE 
countdown indicates the amount of time early or late, 
relative to the scheduled time, an aircraft would be 
if it were presently issued the turn instruction. The 
time difference between the SLT (scheduled landing 
time) and the ETA is the DICE value displayed. A 
straight clock countdown indicates the time remain- 
ing before the turn is to be issued. The difference 
between the computed time to issue a turn and the 
current time would be displayed in the straight clock 
countdown. 



The DICE countdown technique (category E5 
with either Cl or Dl) was selected as the time di- 
rect aid to evaluate for two reasons. First, it gives 
controllers direct information relative to their action 
and the desired end goal time schedule with path 
geometry factored in. If, for example, prior to a 
downwind- to-base turn, the controller wanted to re- 
duce separation by so many seconds, DICE count- 
down gives that information directly. The second 
reason is that the straight clock countdown is a sim- 
ple and common experience for everyone. There- 
fore, it would be more valid to evaluate DICE and 
then ask for a comparison or preference between the 
two types of countdown rather than the other way 
around. Subject responses to countdown preference 
is presented in section 5. 5. 1.4. The DICE countdown 
format is described in detail in section 3.2.3. 

3.1,5. Approach Speed Issues 

It is common practice at high density TRACON’s, 
during heavy-demand IMC periods, to slow the 
traffic entering the final-approach region to a sin- 
gle pattern speed (typically 170 knots) for aircraft 
performance compatibility between transport and 
commuter aircraft and for more planning time to 
organize the traffic. At other times, a higher pat- 
tern speed (typically 190 or 210 knots) is used to 
the base leg and then reduced to the slower pattern 
speed before the final turn. In both cases, the con- 
trollers normally fine-tune their separations on the 
final-approach course (hereinafter called final), via 
their base-to-final turn. In all cases, the controllers 
check for compliance and ensure safe separation on 
final, but generally refrain from exercising any signif- 
icant amount of control action after the final turn. At 
less heavily loaded terminals or at high density ter- 
minals during low-demand periods, aircraft are often 
kept at a higher pattern speed until turned to final 
and then slowed to the lower pattern speed. Airlines 
and pilots generally prefer higher speeds closer in be- 
cause of the reduced delay and because of the higher 
fuel consumption resulting from high drag configura- 
tions required at slower pattern speeds. 

In the FAS A study, we have taken as our exper- 
imental condition the two representative conditions 
described above. The 170-knot approach speed pro- 
cedure simulates the single pattern-speed, IMC case. 
The 210-knot approach procedure normally main- 
tains aircraft at 210 knots through the base-to-final 
turn after which the timing of a 210- to 170-knot 
speed reduction is used to further fine-tune 
separations on final. 

For the situation where aircraft are slowed to 
170 knots before being turned to final, the principal 


issue is accuracy of turn control. Assuming the base 
legs are flown as expected, the correct timing of the 
base-to-final turn is the main factor in determining 
aircraft separation precision. The major concerns in 
this case are: (1) the precision of the controller’s un- 
aided judgment in timing the final turn for separation 
from preceding aircraft (manual format), (2) the pre- 
cision of the controller’s judgment in timing the final 
turn for merging with the aircraft’s own slot marker 
(centerline slot marker format), and (3) the precision 
of the final turn with the help of the two direct au- 
tomation aids (graphic marker and DICE countdown 
formats). 

The issues relative to the 210-knot procedure are 
somewhat more complicated and related to local 
practice and traffic load. Using the modest control- 
lability available from timing a speed reduction on 
final has the potential to further fine-tune the sepa- 
ration precision achieved by turn control. However, 
in some facilities, there is a tendency not to routinely 
apply control after the base-to-final turn. Also, there 
seems to be more application of speed control, on 
final during IMC, at terminals where a straight-in 
arrival route is merged with other routes. 

The centerline slot marker concept (ref. 12) was 
developed at MIT and studied there by using a 
Boston terminal area simulation with a two-speed 
pattern profile, similar to the 210-knot procedure. 
Under those conditions, the aircraft’s centerline slot 
markers were used as a guide to time the speed re- 
ductions. Similarly, a direct (graphic or DICE) au- 
tomation aid can also be employed to indicate when 
or where on final the slower pattern speed should be 
issued to fine-tune aircraft separations. Both speed- 
reduction aids were incorporated into the Langley 
TIMER (traffic intelligence for the management of 
efficient runway-scheduling) simulation (ref. 6) to ex- 
amine performances. Note that for the 210-knot pro- 
cedure, the direct speed reduction advisories on final 
are in addition to the earlier described direct turn 
advisories. The combination of the turn and speed- 
reduction aids also raises the question of clutter and 
distraction in addition to the acceptance of the basic 
procedure of issuing speed reductions on final. 

3.2. Description of Final- Approach 
Spacing Aids Formats 

The significant features of the arrival routes in 
the terminal area of the Denver Stapleton Interna- 
tional Airport that are modeled in the TIMER sim- 
ulation (described in section 4.1.2) are depicted in 
figure 1. The arrival routes are a basic four-corner- 
post structure merging to a single approach path 
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to runway 26L. Of interest in this study is the fi- 
nal controller’s airspace which includes the western 
approach paths from abeam the airport eastward 
and the eastern base and final- approach legs. More 
complete discussions of the airspace and procedures 
simulated are contained in section 4.2. 

All arriving aircraft speeds are reduced to the ap- 
propriate pattern speed (170 KIAS or 210 KIAS) be- 
fore the aircraft enter the final controller’s airspace. 
The TIMER algorithms compute ETA’s at the run- 
way threshold, for aircraft approaching the final con- 
troller’s airspace, based on nominal flight paths and 
speed trajectories. In the 170-knot procedure, a con- 
stant indicated airspeed (170 knots) is assumed up to 
the final-approach fix (FAF). In the 210-knot proce- 
dure, the higher indicated air speed (210 knots) is as- 
sumed to a specified distance from the FAF followed 
by the slower speed (170 knots) to the FAF. Aircraft 
within the fine-tuning region are allowed more lati- 
tude in flight paths to accommodate cumulative flight 
errors and schedule changes. This increased latitude 
is initiated at the last fixed-point update which is 
approximately abeam (abreast) the active runway 
threshold for western arrivals and just outside of the 
FLOTS and WIFES intersections for eastern arrivals. 
At these points, the location in the arrival sequence 
of an aircraft crossing the point is frozen, a target 
scheduled landing time (SLT) is computed, and, for 
eastern arrivals, an appropriate base-leg heading is 
assigned. At a point abeam the FAF for western air- 
craft on downwind legs and at 6.5 mini, from the ex- 
tended runway centerline for eastern aircraft on base 
legs, the target SLT is again computed. These val- 
ues are held constant for the remainder of the ap- 
proach except that when operating in a nonmanual 
(i.e., computer-aided) mode, the target SLT may be 
forced backward or allowed to slip forward (within 
limits) by changes in the ETA’s of preceding aircraft. 

The final-approach spacing aids are then applied 
to the remainder of the approach up to the FAF. 
In addition to a non-FASA or manual mode of op- 
eration, there are three FASA concepts integrated 
into the TIMER. These are centerline slot marker, 
DICE countdown, and graphic marker formats. The 
centerline slot marker concept is based on the differ- 
ence between the scheduled time of arrival (STA) at 
the FAF and the current time of day. The latter two 
concepts are based on the difference between the SLT 
and current ETA at the active runway threshold. 

3.2.1. Manual Operation Mode 

Although SLT’s are generated in the feeder sec- 
tors to provide an organized traffic flow, in man- 
ual operation, no advisories are generated to main- 


tain separation in the final- approach region. Only 
the normal ARTS III data block is provided which 
contains aircraft identification, altitude, and ground 
speed. The controller may also select a display 
of aircraft type which is time shared with the 
altitude/ground-speed line of the data block. As 
the name implies, the controller is expected to use 
current-procedure manual vectoring techniques and 
speed adjustments to maintain the required sepa- 
rations. This mode provides a baseline or calibra- 
tion of the ability of the test subject controllers to 
space and turn aircraft to achieve proper interarrival 
separation. 

An example display scenario, taken from one 
of the 210-knot approach-pattern-speed data runs, 
is depicted in figure 2. Here, “Continental 101 
(COA101)” is descending through 6500 ft at 
140 knots ground speed (GS) to runway 26L. 
“Continental 533” is about 1.2 n.mi. from the FAF 
at 7200 ft and 170 knots GS. “Delta 989” is in 
the process of intercepting the ILS for runway 26L 
5.5 n.mi. from the FAF and is still at 210 knots 
IAS (200 knots GS due to headwind). “TWA 896” 
is on a base leg from the south and descending 
through 11100 ft to 8000 ft at 210 knots IAS 
(240 knots GS). “United 280 and 720” are on their 
downwind legs at 210 knots IAS. “Delta 971” is ap- 
proaching from the southeast with a pending hand- 
off from feeder to final- approach control. Its en- 
tire data block would be flashing on the display of 
the final- approach controller, to indicate the initia- 
tion of a handoff. Approaching from the northwest, 
“United 305” is still under feeder control. 

3.2.2. Centerline Slot Marker Format 

Figure 3 presents the same traffic scenario as in 
figure 2, but with the centerline slot marker for- 
mat displayed on the final controller’s display. The 
circular centerline slot marker symbol of approxi- 
mately ^4 n.mi. diameter (at the scale selected for 
the final- approach controller display) is centered on 
the extended runway centerline as a target mov- 
ing at a ground speed that is equivalent to flying 
active-runway heading, 170 knots IAS, at an altitude 
of 7200 ft. The markers arrive at the FAF at the 
time associated with the STA of the corresponding 
aircraft. The three-digit flight number of the aircraft 
associated with a slot marker is displayed in the cen- 
ter of the marker whenever the aircraft coordinates 
are outside the marker symbol. Slot markers are not 
displayed between the FAF and runway. 

The final controller’s task is to direct the aircraft 
to a merger on the radar display with its correspond- 
ing slot marker. Although the landing sequence and 
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spacing is displayed in a position/distance format, 
there is no explicit “when to turn” indication. In the 
170-knot pattern-speed approaches, the controller is 
expected to use vectoring to guide the aircraft to 
the center of its slot marker. When operating with 
210-knot approaches, the controller is expected to 
issue a speed reduction to 170-knots IAS after the 
aircraft is turned to an ILS-intercept heading such 
that the aircraft position symbol is in the center of 
the slot marker when the speed reduction is complete. 

The differences in slot marker presentation be- 
tween the 170- and 210-knot approach-pattern-spccd 
procedures are minimal. In the 170-knot version, 
it is assumed that the aircraft will fly the assigned 
speed to the FAF. If the controller chooses to close 
a gap in the traffic that has developed in the final- 
approach airspace (in the 170-knot version), an in- 
crease in airspeed may cause the SLT of the subject 
aircraft as well as the SLT’s of the trailing aircraft 
to slip forward (up to their forward-slippage lim- 
its). In the 210-knot version, the SLT of an air- 
craft is not changed by speed adjustments because 
it is assumed that speed adjustments will be used 
to mate the aircraft with its marker (i.e., attempt- 
ing to match the ETA with the SLT). However, the 
SLT’s of trailing aircraft may be affected in order to 
maintain separation with an ETA that results from 
an overcompensated speed adjustment. 

In the 210-knot scenario depicted in figure 3, 
“Continental 533” is somewhat late as indicated by 
the relative positions of its slot marker and the air- 
craft symbol. “Delta 989” is in the proper posi- 
tion to begin decelerating from 210 to 170-knots 
IAS. “TWA 896” is probably within a mile of 
where its ILS-intercept vector should be issued and 
“United 280” should be vectored from downwind-to- 
base within about 2 n.mi. to follow TWA. A landing- 
order-sequence list all of the aircraft in the terminal 
area is provided in the upper-right corner of the dis- 
play. This list is intended to aid the controller in 
anticipating the landing sequence of aircraft whose 
slot markers are not yet in the field of view. 

3.2.3. DICE Countdown Format 

Figure 4 presents the same traffic scenario as in 
figure 2, but with the DICE countdown format dis- 
played on the final controller’s display. The DICE 
countdown format for vector turns consists of dis- 
playing the DICE countdown value and the rec- 
ommended heading of the next flight-path segment, 
within the aircraft data block. The DICE count- 
down format for speed reductions in the 210-knot 
procedure consists of displaying the DICE count- 
down value and the nominal final pattern speed 


(170 knots). These parameters are displayed on tw T o 
subsequent lines below the altitude/ground speed in 
the data block. The DICE value displayed is the dif- 
ference between the periodically updated ETA (re- 
computed every radar scan for aircraft in the final- 
approach fine-tuning region) and the SLT of a given 
aircraft. Thus, the DICE value for a vector turn in- 
dicates how early, relative to its SLT, the aircraft 
would be if its turn instructions were issued imme- 
diately. The. display of this information is withheld 
until the DICE value is less than a prescribed thresh- 
old. The thresholds w r ere chosen to turn on the DICE 
value display when the aircraft is within 45 sec of an 
anticipated turn in the 170-knot procedure. Reason- 
ing that a larger lead time should be provided in the 
higher speed procedure, a threshold of 60 sec was 
chosen for the 210-knot procedure. Expected mes- 
sage length and controller/pilot response time are 
factored into the displayed DICE values. 

The change in ETA (and consequently in DICE 
value) per unit time (i.e., ETA gain) is about 2 for 
aircraft on downwind and about 1 on base legs. The 
gain is about 2 on the downwind because, for each 
mile the downwind is extended, the aircraft must fly 
a corresponding extra mile on the final. As a point 
of interest, the ETA gain for the speed reduction 
from 210 to 170 knots on final approach is only 
0.25 sec/sec. That is, a 1-sec delay in implementing 
the speed reduction w^ould only result in x k sec earlier 
arrival time. 

When the DICE value becomes less than zero (the 
time w T hen the controller should issue the advisory), 
the recommended value (heading or speed) flashes to 
further attract the controller’s attention. The infor- 
mation is extinguished from the data block after the 
appropriate change in heading is initiated. Because 
of radar- tracker instability, DICE values are not dis- 
played during heading- or speed-change transitions 
until tracking has restabilized. 

During a turn from a dowmwdnd leg to a base leg, 
a determination is made as to whether a base leg 
that will allow sufficient time to generate a normal 
turn to intercept the ILS can be expected. If so, the 
DICE display is resumed as soon as the tracker stabi- 
lizes at the end of the turn to base. However, if little 
or no base leg is predicted, the system provides a 
temporary, linear, artificial countdown to assist the 
controller until the tracker stabilizes and generates 
more accurate DICE values. A significant change 
can occur in the displayed DICE value when the 
tracker settles due to discrepancies between the pre- 
dicted and actual flight paths during the turn. No 
recommended heading is displayed during the arti- 
ficial countdown. Because of larger turn radii, the 
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artificial countdowns are generated much more often 
during the 210-knot operations than they are during 
the 170-knot operations. 

3. 2. 3.1. DICE for 170-Knot Operation 

In the 170-knot approach-pattern-speed proce- 
dure, the DICE countdown values are displayed 
whenever they are less than 90 sec for downwind-to- 
base turns and less than 45 sec for base-to-final turns. 
These threshold values give the controller about the 
same amount of lead time for both cases since the 
ETA gain on the downwind legs is about 2.0 and 
for the base legs is about 1.0. The criteria for dis- 
playing the recommended heading are that the DICE 
value must meet its display threshold criteria and 
must also be less than 60 sec. The heading value 
that is displayed is the heading used in the DICE 
ETA calculation. That is, a base-leg heading is the 
prescribed base-leg ground-track angle corrected for 
the wind expected at the base-leg altitude. An ILS- 
intercept heading is based on a 20° intercept, a turn 
directly onto the localizer, or intercept angles in 5° 
increments if the aircraft is in an overshoot situa- 
tion. In the 170-knot version, there is no adjustment 
to the threshold when an overshoot (of the extended 
runway centerline) is anticipated. Because the DICE 
parameters are not displayed until the thresholds are 
met, an aircraft that is predicted to have an ex- 
tremely large overshoot may be quite close to the 
final- approach course before the DICE is displayed. 

3. 2. 3. 2. DICE for 210-Knot Operation 

In the 210-knot procedure, the threshold, for dis- 
playing the DICE heading value, is based on pro- 
jected time to go (clock time) to the turn point rather 
than on the DICE countdown value. The threshold 
value was set to 60 sec for all DICE turns. The time 
to go to the turn point to satisfy an SLT is the DICE 
value divided by the ETA gain. The other major dif- 
ference with respect to the 170-knot procedure is in 
the display of final-approach-course overshoot situ- 
ations. This version displays the DICE parameters 
as soon as an overshoot is predicted and also shows 
the ILS-intercept heading that will be needed when 
the time-to-turn point is reached. Not only does this 
provide more lead time but also provides the con- 
troller better information on the extent of the pre- 
dicted overshoot than in the 170-knot version. Thus, 
the controller has the option to modify an aircraft’s 
profile on the base leg to reduce or eliminate the nor- 
mally operationally undesirable overshoot. Examples 
of normal (nonovershoot) DICE value displays are 
shown in figure 4. The data block for “TWA 896” 


shows a recommended ILS-intercept heading of 280° 
with a DICE value of 13 sec. The data block for 
“United 280” shows a recommended base-leg heading 
of 345° with a DICE value of 59 sec, which should 
count down to zero in about 26 sec. 

When the TIMER detects that a turn to an ILS- 
intercept heading has been initiated, it computes the 
time at which the turn should be completed. This 
time is then used to activate the speed-DICE pro- 
cedure which advises the controller when to reduce 
the aircraft speed to 170 knots during the remain- 
ing final-approach flight segments prior to the ap- 
proach gate (a point 1 mile outside the FAF or 5 miles 
from the runway threshold, whichever is farther). It 
should be noted that “early turn” and “early speed 
reduction” have the opposite effect on aircraft arrival 
time with respect to the SLT. Thus, the DICE value 
indicates how late the aircraft will be with respect 
to its SLT if a reduction to the recommended IAS 
(170 knots) is issued immediately rather than how 
early as in the case of vector-DICE. A more complete 
discussion of the vector- and speed-DICE algorithms 
and display criteria is contained in appendix A. 

Like a vector-DICE, the speed value flashes when- 
ever the speed-DICE value is less than or equal to 
zero. Again referring to figure 4, the data block for 
“Delta 989” shows a speed-DICE value of zero indi- 
cating that the speed reduction to 170 knots should 
be issued. In this case, the data block field show- 
ing the recommended target speed “SI 70” would be 
flashing. 

3.2.4. Graphic Marker Format 

The graphic turn and speed reduction (in the 
210-knot TIMER version) FAS A provides a pictorial 
representation of where, rather than when, to initiate 
a flight profile (heading or speed) change. Figure 5 
presents the same traffic scenario as in figure 2 but 
with the graphic marker format displayed on the final 
controller’s display. The controller’s objective is to 
initiate the change as the aircraft symbol comes in 
contact with the graphic marker. This is intended to 
be coincident with the DICE value counting down to 
zero. 

The display thresholds for graphic turn markers 
are the same as for DICE heading-vector values. A 
turn marker is displayed to the final controller when 
an aircraft is within a flight time to the predicted 
turn point of about 45 sec for 170-knot procedure 
and 60 sec for 210-knot procedure. The turn mark- 
ers consist of three directed line segments. Although 
only two segments are required to depict the neces- 
sary information, the three-segment marker avoids 


Tl 


8 



the exaggerated segment lengths of the two-segment 
marker for turns greater than 90° , as discussed in 
section 3.1.4 on FAS A issues. The first segment 
is oriented along the projected flight path directly 
ahead of the aircraft. Its length is determined by the 
expected ground speed of the aircraft and the nom- 
inal times expected for controller communication to 
the pilot, pilot response, and roll-in to the turn. The 
second and third segments complete an approxima- 
tion of a constant-radius turn with the third segment 
aligned with the heading necessary to maintain the 
ground track of the next flight leg. As in the DICE 
FASA when an artificial DICE countdown is gen- 
erated, an early-warning ILS-intercept turn marker 
will be generated to assist the controller when a very 
short (or nonexistent) base leg is computed during 
a downwind- to- base turn. Again, this occurs more 
frequently at 210 knots than at 170 knots. 

The only difference between turn markers in the 
170-knot and 210-knot operations, other than in 
physical size due to different ground speeds, is that 
the orientation of the third segment of base-to-final 
turn markers always represents a turn from current 
position to intercept the ILS in the 170-knot opera- 
tions but represents the expected ILS-intercept head- 
ing in the 210-knot version. This is relevant only in 
predicted overshoot situations where the turn marker 
extends beyond the localizer centerline. When this 
situation is detected, the base-to-final turn marker is 
presented to the controller immediately and may in- 
dicate a projected turn directly onto final to as much 
as a 35° intercept from the opposite side of the local- 
izer, depending on the magnitude of the anticipated 
overshoot. 

Speed- reduction markers, in the 210-knot TIMER 
version of the graphic marker format, are small cir- 
cles enclosing an x. The graphical speed reduction 
symbol used is a monochrome version of the speed 
symbol in reference 15, which they employed to indi- 
cate speed reduction points on the approach routes 
prior to final. A display threshold limit of 3.0 n.mi. 
ahead of the aircraft is applied to the speed mark- 
ers to prevent display ahead of a preceding aircraft. 
This reduces clutter and prevents possible confusion 
as to which aircraft the speed reduction was intended. 
Also, whenever a speed marker is predicted to be dis- 
played on the opposite side of the extended runway 
centerline from the aircraft to which it belongs, it is 
projected onto the centerline. 

Examples of both types of graphic markers are 
shown in figure 5. “Delta 989” should be given its 
speed reduction to 170 knots immediately, whereas 
“TWA 896” and “United 280” still have some dis- 
tance to go before being vectored to ILS-intercept 


and base-leg headings, respectively. Note that a 
speed-reduction marker is extinguished as soon as 
the speed change is initiated. For turn markers, the 
marker position is “frozen” just before the aircraft 
reaches the recommended turn point or when the 
appropriate change in heading is initiated and the 
marker remains visible until the time projected to 
complete the turn has elapsed. 

Figure 6 presents an edited portion of the fig- 
ure 2 210-knot traffic scenario with a graphic marker 
showing an extended runway centerline overshoot, if 
normal procedure is followed. Here, an ILS inter- 
cept heading of 240° is indicated for “TWA 896.” 
Note, the runway heading was 260°, and the display 
was oriented in a magnetic north-up configuration. 
A more in-depth discussion of the graphic marker 
algorithms is also included in appendix A. 

4. Experimental Facilities and 
Conditions 

4.1. Experimental Facilities 

A sophisticated real-time ATC simulation using 
the Denver Stapleton approach routes provided a 
realistic, dynamic environment to assess final con- 
troller performance with the FASA display formats 
tested. A real-time version of the terminal-area 
air traffic model (TAATM), with the TIMER al- 
gorithms embedded (section 4.1.2), was interfaced 
with an air traffic controller station to provide the 
traffic to be controlled and the final- approach spac- 
ing aids (FASA) studied. The computer data inter- 
faces, voice links, and controller workstations of the 
Langley Mission-Oriented Terminal-Area Simulation 
(MOTAS) Facility (section 4.1.1) interactively linked 
the TIMER algorithms, the subject controller, and 
the pseudopilot station operators. These features will 
be briefly described in the following paragraphs. 

4.1.1. Mission-Oriented Terminal- Area 

Simulation 

The Langley Mission-Oriented Terminal-Area 
Simulation (MOTAS) Facility (ref. 17) provides a 
flexible representation of the airborne, ground-based, 
and communication aspects of the ATC terminal- 
area environment. The elements of the MOTAS facil- 
ity used in this experiment are shown in figure 7 . The 
elements included a terminal ATC model, aircraft 
models, pseudopilot stations, air traffic controller 
stations (section 4.1.3), and a simulated air /ground 
communication network. 


9 



All the final approach sector traffic in this study 
was “flown” via a pseudopilot station. The sim- 
ulated aircraft were controlled by keyboard input 
of commands which dictated functions such as air- 
speed, altitude, heading, interception of the ILS. 
In addition, pilot verbal radio communication to 
ATC were provided. To prevent overload and 
ensure representative pilot- aircraft responses, the 
pseudopilot station servicing the final control sec- 
tor was staffed by a two-person team (fig. 8). One 
person was the keyboard operator who input, in a 
timely manner, the commands to control the air- 
craft traffic. The other person was the radio oper- 
ator who simulated the radio communication of all 
aircraft entering and flying through the final sec- 
tor. This operator was trained to use proper phrase- 
ology and radio protocol. To enhance the realism 
for the subject controller, the radio operator used 
a voice disguiser to impersonate the voice quality 
of multiple pilots. In addition to their primary 
keyboard and radio operator duties, each pseudo- 
pilot team member monitored the companion’s out- 
put to ensure accuracy and realism. 

In addition to controller comments to that effect, 
another indication of the realism and fidelity of the 
experimental simulation is given in figure 9. Shown 
is a histogram of actual pilot response times to ATC 
turn commands from reference 11 and a histogram 
of the pseudopilot response times recorded in this 
experiment. 

4.1.2. TIMER Simulation 

The time values and display locations for each of 
the FASA formats evaluated were obtained from the 
TIMER (traffic intelligence for the management of ef- 
ficient runway-scheduling) algorithms. TIMER is an 
extended-terminal-area, computer-aided, time-based 
ATC concept which has many features similar to the 
NASA Ames CTAS system (refs. 1 through 3) which 
will be used in the FAA’s Terminal ATC Automation 
(TATCA) Program. The major operational features 
of the TIMER concept arc shown in figure 10 and 
were applied to the Denver Stapleton nominal ap- 
proach pattern for a land-runway 26L configuration 
shown in figure 1. The principal TIMER model fea- 
tures are summarized as follows: 

1. The arrival traffic stream into the extended ter- 
minal area is derandomized at the horizon of con- 
trol by establishing a proposed aircraft landing se- 
quence and building a list of aircraft SLT’s based 
on standard ATC separation criteria (events (j) 
and ©of fig. 10). The desired metering-fix time 
as a result of the assigned landing time is also 
determined. 


2. Nominal estimated times of arrival used in fea- 
ture 1 are based on aircraft performance models. 
From using these models and predicted winds, 
a ground-computed, fuel-saving, profile-descent 
trajectory, to the aim point, is determined to meet 
the aircraft ’s assigned SLT (events © and 0 of 
fig. 10). 

3. Computer- generated assistance is given to the 
en route controller to help in meeting aircraft tar- 
get times based on the en route cruise and pro- 
file descent trajectory calculations. The param- 
eters determined are the en route cruise speed, 
the time to initiate and the Mach/CAS speeds to 
fly a flight-idle- thrust descent, and the terminal 
segment speeds and headings. 

4. Adjustments to the SLT’s and, if necessary, 
changes in the landing sequence are made to ac- 
commodate errors and anomalies in factors such 
as wind, navigation, airspeed, and heading which 
affect the SLT of either the own aircraft or the 
preceding aircraft. These schedule adjustments 
occur at the following points shown in figure 10: 
the metering fix, the speed adjustment points, 
and the fine-tuning region. The landing sequence 
is fixed and the aircraft speeds are reduced to 
the appropriate pattern speed (170 or 210 knots) 
before aircraft arrive in the fine-tuning region. 

5. The aircraft trajectory is fine-tuned in the final- 
approach region in order to meet the aircraft’s last 
adjusted SLT with a minimum time error. This 
is where the FASA indicated turn and speed ma- 
neuvers occur. The TIMER fine-tuning region is 
defined by the boundaries of the vector headings 
from the aim point for eastern arrivals (event ©in 
fig. 10) and by the boundaries of the downwind- 
to-base turn for the western arrivals (event © in 
fig. 10). The dashed lines in both figures 1 and 10 
indicate the boundaries of the fine-tuning region. 
Within this region, a computer-aided fine-tuning 
maneuver consists of timing when or locating 
where both the turn-to-base (event © in fig. 10) 
and the turn-to-final (event ©) maneuvers should 
occur. An additional fine-tuning feature was 
added for the 210-knot approach procedure (de- 
scribed in sections 3.1.5 and 3.2). This consists 
of timing when or locating where, on final, the 
speed reduction from 210 to 170 knots should 
occur (event ©in fig. 10). In all cases, a compari- 
son of the aircraft’s SLT with its estimated time of 
arrival (ETA) forms the basis for either the time 
value (the when) or the position (the where) of the 
FASA formats described in section 3.2. More de- 
tails of the SLT and ETA computation are given 
in appendix A. 



A more complete description of the TIMER con- 
cept and model algorithms is furnished in reference 6. 
In this study, the en route portion of the TIMER con- 
cept was only modeled in the time dimension (i.e., 
4-D aircraft trajectories were not calculated; how- 
ever, pertinent en route events such as initial sched- 
uling, delays, and metering-fix time errors were com- 
puted). From the metering fix to the runway (the 
nominal terminal region) the entire aircraft trajecto- 
ries (4-D paths) are deterministically modeled with 
airborne- flight-technical and ground-radar errors in- 
cluded. The ent ire extended-terminal TIMER model 
was operated in the study; however, only the region 
inside the metering fixes had live, direct controller 
interaction. Like the CTAS Final- Approach Spacing 
Tool (FAST), the TIMER simulation accommodated 
controller issued headings and speeds which deviated 
from those used in a normal traffic pattern. This ro- 
bustness added to the realism of the simulation and 
to the validity of the results. 

4.1.3. Simulation Air Traffic Controller 

Station 

Real-time air traffic controller interaction with 
the TIMER simulation and the pscudopilots was ac- 
complished via the two monochrome displays, sim- 
ulation controller stations of the MOTAS facility. 
As shown in figure 11, the two controller stations 
(ATC station 1 on the left and station 2 on the 
right) were equipped with a plan position indicator 
(PPI) on which the TIMER-simulated aircraft posi- 
tion information was displayed. Station 1 (fig. 12) 
was of primary interest because it was designated 
as the final controller station and used by the FAA 
subject controllers. A coordinator was provided 
to assist the subject controller in performing non- 
control functions and was seated to the immediate 
right of the subject. Station 1 had a communica- 
tion system for radio communication between the 
subject controller and the pseudopilot station servic- 
ing the final sector. Standard FAA type push-to-talk 
headsets, Plantronics model HS 0110-2E, were used 
with an optional foot-pedal switch. In addition, sta- 
tion 1 was equipped with an oculometer electro-optic 
head to obtain controller lookpoint data (described 
in section 4.1.4). 

Station 2 was used by the in-house controller 
(former FAA terminal controller) who simulated the 
feeder function for all the experimental runs. This 
controller only modified aircraft headings, altitudes, 
or speeds in his sector if requested by the subject 
final controller, or if the feeder himself judged it nec- 
essary. (During the test, feeder modifications were 
only necessary during some manual, no FASA, runs 


because the final became too extended.) Follow- 
ing acceptance of the handoff by the final controller 
(discussed in section 4.2.2), transfer of communica- 
tion was initiated by a keyboard entry at station 2 
which prompted the pseudopilot to perform a radio 
“check-in.” 

The controller displays were simulated by an 
Evans & Sutherland Multi Picture System with a 
CRT measuring 23 in. in diameter and mounted ver- 
tically (figs. 11 and 12). The displays emulated a 
digitized radar presentation and were each configured 
with an appropriately scaled video map of the Denver 
terminal area (fig. 13). The latest radar-derived, 
aircraft locations were indicated by the letter sym- 
bol which corresponded to the control position re- 
sponsible for the aircraft. The three past locations, 
at earlier radar scans, were represented by three 
“dots” of decreasing intensity. These dots convey 
trend information since no “trail” information was 
available with the simulation display. Each aircraft 
location was accompanied by its normal ARTS III 
alphanumeric data block and a FASA display when 
appropriate (section 3.2.1). 

4.1.4. Oculometer Facility 

The remote oculometer at Langley is a highly 
modified Honeywell Mark 3A system used to deter- 
mine and record the lookpoint of a test subject. Some 
of its defining features arc: 

The system is unobtrusive and nonrcstrictive; 
it projects a beam of collimated near- infrared 
light at the subject’s eye and then processes the 
reflected return to a video camera 

Both hardware and real-time software have been 
modified at Langley to better accommodate the 
air traffic control environment; postprocessing 
software was developed in-house to correlate look- 
point data with the corresponding objects on the 
controller’s display 

The system tracks head movement in a 1-ft cube 

It quickly reestablishes track when subjects rotate 
their head and then turn back toward the display 

The sample time is every 1/30 sec; it writes a data 
record at the end of each fixation or out-of-track 
event (defined later) 

The system is capable of video taping the data 
runs; the lookpoint, which is electronically super- 
imposed on the PPI scene, is recorded as well as 
the final controller voice channel and the control 
room audio 

The following discussion expands on some of the sys- 
tem features. However, more detailed information on 
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the oculometer system can be found in section E.L 
In addition, appendixes A and B of reference 18 con- 
tain a detailed description of the Langley oculometer 
system and its use in cockpit display evaluation. 

The oculometer facility computes and stores a 
time history of eye-scanning events. These events 
characterize the interaction of the controller-subject 
with the display as he or she directs air traffic in the 
simulated TRACON facility. Each event is either in- 
track or out-of-track. In- track events are lookpoint 
fixations which are characterized by a continuous 
focus of the subject’s gaze on a small area of the 
display (usually in the vicinity of a display object) 
for multiple oculometer sample periods. Dwell time 
is defined as the length of a fixation. An out-of- track 
event or a significant movement of the lookpoint 
causes a fixation to end and triggers the recording 
of the event. Out-of- track events occur when the 
system loses track of the subject’s eye for some period 
of time. This track loss occurs during blinking or 
if the controller turns his head for various reasons. 
An out-of-track ends with an in-track, that is, the 
recapture of the eye. For each in-track fixation event, 
the system records 

The location of the controller’s lookpoint on the 

radar display 

The pupil diameter 

The duration of the lookpoint fixation 

The system records an event as a single file record 
with the duration of the event being an integer 
multiple of the oculometer sample period. During 
postprocessing, out-of- track events are classified as 
noise when less than 4 periods in duration, as a blink 
when between 4 and 12 periods, and as being “long” 
(i.e., turning head or blocking beam) wffien greater 
than 12 periods. 

4.2. Experimental Conditions, 

Controller Task, and Subject Profile 

4.2.1. Terminal Area Conditions 

Simulated 

The arrival routes of Denver’s Stapleton Inter- 
national Airport operating in a “land-runway 26L” 
configuration (fig. 14) were simulated assuming IMC. 
The ILS runway 26L approach plate is shown in fig- 
ure 15. A linear wind model using statistical coef- 
ficients for an average Denver area wind was used 
in all runs. The simulated wind velocity at ground 
level was 7.9 knots from 277° with a speed gradient 
of 2.37 knots per 1000 ft. In this study, the wind 
direction was constant at all altitudes. 


The arrival traffic simulated was made up entirely 
of airline transport aircraft. The route- loading and 
aircraft-type distribution were obtained from the Of- 
ficial Airline Guide (OAG) information and used to 
construct computer-generated traffic samples repre- 
senting weekday arrivals at Stapleton. All the traf- 
fic entered the terminal area at the metering fixes. 
The metering fixes at Stapleton are KEANN, IOC 
(Kiowa), BYSON, and DRAKO (figs. 1 and 14). The 
simulated arrival traffic was distributed at the four 
metering fixes in the following manner: 


Metering fix 

Percentage of total traffic 

KEANN 

24 

IOC (Kiowa) 

29 

BYSON 

26 

DRAKO 

21 


The traffic arrival rate was between 39 and 42 air- 
craft per hour depending on the mix of large and 
heavy aircraft for the particular traffic sample used. 
The long-term average traffic mix was 87.5 percent 
large aircraft and 12.5 percent in the heavy cate- 
gory. Large aircraft are those with a maximum certi- 
fied takeoff weight of between 12 500 and 300 000 lb. 
Heavy aircraft are those capable of takeoff weights 
of 300 000 lb or more regardless of their actual weight 
(ref. 19). 

Inside the FAF, the separations briefed to the 
subjects and used by the FASA algorithms were a 
function of the weight class of both the lead and 
trail aircraft of a pair. By using aircraft velocities 
and simulated winds, the separation distances were 
converted to separation times for scheduling purposes 
in the TIMER algorithms. The distance separation 
criteria, used in terms of the lead and trail aircraft of 
a pair, conformed to current FA A standards (ref. 19) 
for airports where reduced separations (2.5 n.mi.) 
inside the FAF is authorized. The separations were 
as follows: 



Separation of 


trail aircraft, n.mi. 

Lead aircraft 

Large 

Heavy 

Large 

2.5 

2.5 

Heavy 

5 

4 


4.2.2. Approach Paths and Procedures 

The airspace and procedural environment closely 
resembled that in use at Denver Approach Control. 





The final controller’s airspace (referred to at Denver 
as the dump region) is centered over the final- 
approach course, extending from the approach end 
of runway 26L to a point 20 n.mi. east and 8 n.mi. 
on either side of the final approach course (figs. 13(b) 
and 14). For simplicity in briefing, the dump airspace 
vertical limits of the simulation, surface to 10000 ft, 
were slightly modified from the actual Denver oper- 
ation. The feeder controller’s sector is composed of 
essentially the arrival corridors ( 1 1 000 ft and above) 
and the airspace above the dump region. 

There were two final-sector pattern-speed proce- 
dures modeled in TIMER to evaluate the FASA’s. 
One version assumed a constant pattern airspeed un- 
til the reduction to final- approach speed begins in 
the vicinity of the FAF. In this evaluation, the con- 
stant pattern-speed value used was 170 knots and was 
called the 170-knot procedure . The other TIMER 
version allows a higher initial airspeed through the 
turn-to-final then reduces to a slower pattern speed 
which is maintained until the final- approach speed 
reduction begins. In our evaluation, 210 knots were 
used for the initial pattern speed followed by a re- 
duction to 170 knots after the turn-to-final and was 
called the 210-knot procedure. 

Figure 14 depicts the normal traffic flow in the 
arrival corridors and dump airspace. Speeds for all 
aircraft were reduced, at predetermined points out- 
side the dump region, to an IAS of 170 or 210 knots, 
depending on the speed procedure being used for a 
given scenario. Traffic in the simulation proceeded 
inbound via the four arrival corridors, tracking ra- 
dials to the Denver VOR (DEN, fig. 1), descending 
to 11000 ft at an indicated airspeed of 250 knots. 
Aircraft are automatically placed in hand-off status 
to the final sector resulting in a flashing data block 
on the final controller’s display; the coordinator nor- 
mally accepted the hand-offs (transfers of control) for 
the final controller. Prior to simulating communica- 
tions transfer, aircraft from the western fixes were 
turned to a 080° downwind heading; aircraft from the 
eastern fixes are issued headings based on traffic con- 
ditions and are issued an additional descent to 8000 ft 
(western aircraft are issued descent to 8000 ft by the 
final controller). 

Once the aircraft entered the lateral limits of the 
dump region, the subject final controller was respon- 
sible for sequencing, spacing, and issuing vectors for 
the ILS RWY 26L approach (fig. 15). Sequencing and 
spacing was accomplished using vectors and, in the 
210-knot procedure, a speed reduction was also used. 
During runs for which no aids were provided, the 
subject’s own vectoring and speed control techniques 
were used to space and separate the traffic. However, 


when provided, FASA’s were expected to be used to 
the extent possible. The final controller was also re- 
sponsible for issuing the approach clearance and a 
change to the tower frequency. 

4.2.3. TIMER/Controller Interaction 

There are two levels of control in the simulation: 
program (TIMER control) and manual (controller 
commands). For the FAS A studies, aircraft were or- 
dinarily both ATC controlled and “flown” by TIMER 
from the time the aircraft appeared in the simulation 
(in center airspace) until the completion of a hand-off 
to the final controller (prior to the aircraft entering 
the dump airspace). At this time a keyboard entry by 
the feeder controller freed the aircraft to be pseudo- 
pilot controlled based on final controller issued in- 
structions. While the aircraft were normally un- 
der control of the simulation program in the feeder’s 
airspace, the feeder controller was able to input com- 
mands by keyboard to alter the aircraft heading, 
speed, or altitude if he deemed it necessary or was 
requested to do so by the final controller. 

All aircraft in the dump region were flown by the 
pseudopilot-stations keyboard operator in response 
to instructions issued by the final controller. Af- 
ter the approach clearance had been issued by the 
controller, the aircraft would automatically join the 
localizer and intercept the glide slope when encoun- 
tered. A reduction to approach speed also occurred 
in the vicinity of the outer marker. When the aircraft 
reached a height of 20 ft above the runway, the po- 
sition symbol and associated data block disappeared 
from the display. 

4.2.4. Experimental Task and Controller 

Subject Profile 

The controller subjects were assigned the role 
of the final controller during simulation runs. The 
airspace and procedure responsibilities of the final 
controller were explained in section 4.2.2. The pri- 
mary responsibilities were sequencing and spacing 
inbound aircraft and providing the approach clear- 
ance. Subjects were instructed to use the FASA when 
provided; however, they were briefed that they were 
ultimately responsible for separation of the traffic. 

The 12 subjects for this study were all full perfor- 
mance level (FPL) controllers from Norfolk (ORF) 
Approach Control which is a Level IV (Level V de- 
picts busiest U.S. terminals) radar approach control 
facility. Two subjects were supervisors and two oth- 
ers were staff personnel all of which were current in 
the approach control facility. The total time that 
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the subjects had worked at Norfolk Approach Con- 
trol varied from 3 to 13.5 years; the average time at 
Norfolk for all subjects was 7 years; the average time 
as an FPL controller was 5.25 years with a range of 1 
to 10 years. The total period of radar ATC experi- 
ence among the subjects ranged from 6 to 19 years. 
(This time includes only experience at radar facili- 
ties where the types of ATC services provided are 
reasonably comparable with the nature of the task 
in this study.) Nine of the 12 subjects had been FPL 
controllers at other facilities. 

4.3. Experimental Sequence and 
Measures of Performance 

The practice and data sessions for each subject 
controller covered a period of 4 successive days. The 
first 2 days were devoted to one speed procedure, 
whereas the last 2 days were devoted to the other. 
Initially each subject was briefed on the purpose 
of the experiment, the simulation facility and con- 
ditions, the airspace and procedures, the two pat- 
tern speeds to be tested (170- and 210-knot pro- 
cedures), and the duties and responsibilities. The 
controller was then given a hands-on explanation and 
demonstration of the ATC simulation. This was fol- 
lowed by a practice session using manual control (no 
FASA) which lasted until the subject was comfort- 
able with the procedure and simulation and ready 
for the data run. After a suitable break, the manual 
format data run of 1 hr and 10 min was performed 
and the postrun questionnaire answered. The initial 
pattern-speed procedure used for each subject was 
dictated by the experimental matrix. 

Each of the following 3 half-days consisted of 
a FASA format briefing, a practice session, and a 
data session of 1 hr and 10 min followed by the 
post run questionnaire. The format briefings included 
a detailed description of the format and how it should 
be used. For the centerline slot marker, subjects were 
also briefed on suggested techniques to merge aircraft 
with their slots. The three FASA formats used the 
same speed procedure as the initial manual run. 
The order of the FASA formats was dictated by the 
experimental matrix. Table 1 gives an approximate 
time schedule because the length of the practice 
session depended on the subject feeling comfortable 
and prepared to begin a data-collection session. 

As shown in table 1, the second 2 days consisted 
of the manual briefing, practice, data session, and 
postrun questionnaire followed by the three FASA 
formats using the second pattern- speed procedure. 
The 170- and the 210-knot speed procedures each 
had one practice traffic sample and six data samples 


(table 2) to prevent the controller from remembering 
traffic situations from previous data runs. The exper- 
imental matrix is shown in table 3. The runs 'were 
numbered odd for practice runs and even for data 
runs. Each data run is identified by the format and 
the traffic sample record number from either the 170- 
or 210-knot speed procedure. Note, the order of both 
the speed procedures and the FASA formats were 
randomized from subject to subject to minimize any 
learning or order effects on the experimental results. 

From an ideal, experimental design viewpoint, 
always performing the manual format first might 
raise some question as to whether this favors the 
other formats, relative to a learning effect. The 
contention is that the professional experience of the 
subjects in using the manual format in everyday 
practice more than counterbalanced any concern in 
that regard. In fact, the approach was that the 
manual practice run was really an opportunity for 
the subject to get familiar with the airspace and 
the simulation environment rather than a session to 
gain proficiency and experience in using the manual 
format. This approach was selected because it gave 
us the maximum quality and quantity of data return 
for the subject time available. In addition, recall that 
a fair FASA comparative evaluation was the main 
objective. 

The basis for a comparative evaluation of the 
FASA formats is the several parameters of con- 
troller performance and reaction that were mea- 
sured. The key parameter is the separation precision 
but the other measures add important information 
in the comparative assessment. The experimental 
measures, in the order discussed in this report, were 

1. The precision of aircraft delivery and separation 
in terms of standard deviation of aircraft pair 
interarrival error at the runway threshold 

2. The mean number of vectors (heading instruc- 
tions) per aircraft issued by the final controller 

3. The controller response time to a “direct” FASA 
suggested heading and speed advisories 

4. The subject controller eye-scanning behavior on 
the PPI as measured by the Langley oculometer 
system; these processed data indicate information 
such as areas of concentration, dwell time, num- 
ber of aircraft pair cross checks 

5. The controller’s reaction and acceptance from 
subject questionnaire and debriefing comments 

6. The relative workloads of the FASA formats ob- 
tained from the task load index (TLX) technique 
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5. Results and Discussion of 
Real-Time Simulation 

The previously discussed experimental measures 
were taken of the subject controllers 1 performance 
and reactions while serving as the final controller in a 
real-time, interactive TRACON simulation. For each 
of the two representative pattern-speed procedures, 
data were collected by using the four final controller 
display formats. Based on the approach pattern- 
speed procedure issues discussed in section 3.1.5, it 
was decided that specific interest in only one of the 
procedures, as well as generality of results, would be 
served best by presenting the analysis of each mea- 
sure in three subsections. The first subsection dis- 
cusses the 170-knot pattern-speed procedure results, 
the second presents the results of the 210-knot proce- 
dure with speed control on final, and the third com- 
pares the results of the two procedures. This sec- 
tion presents a detailed discussion and data analysis; 
major results are given in section 6. 

5.1. Delivery-Time Precision at Runway 
Threshold 

The measure used to assess control precision is 
the standard deviation of aircraft-pair interarrival 
time error (IAE), which is defined in appendix B. 
Briefly, IAE is the difference in the arrival time errors 
of a pair of sequentially landed aircraft. This error 
can be written in terms of aircraft actual time of 
arrival (ATA) and scheduled landing time (SLT) at 
the runway threshold for aircraft 1 and 2 of the 
sequential pair: 

IAE - (ATA 2 - SLT 2 ) - (ATAj - SLTi) (1) 
By regrouping terms, IAE can also be expressed as 

IAE = (ATA 2 - ATAB - (SLT 2 - SLTi) 

= (Actual separation time) - (Scheduled separation time) (2) 

Even though each aircraft of a pair has its own 
runway threshold time error, if the time error is the 
same for both (i.e., a constant bias), the IAE will 
be 0 and the pair separation will be correct. Thus, 
spread of the IAE’s indicates variation in desired 
spacing and is the attribute of interest. IAE spread is 
characterized by the statistical measure of dispersion 
about the mean, the variance, or its square root, the 
standard deviation. 

When aircraft are on instrument approaches dur- 
ing manual control, the controller concerns are with 


aircraft-pair separations conforming to the radar sep- 
aration standards in effect. Therefore, in the manual 
runs, only separations were measured and the land- 
ing order or scheduled landing times were not kept 
track of. The test subject controllers were instructed 
to aim for minimum allowable separation. Foi the 
tightly packed traffic used in the test, we assume the 
controller was attempting to maximize the landing 
rate by keeping aircraft-pair separations to the min- 
imum distance allowed. For this case, we can treat 
the minimum required separation as the intended or 
scheduled separation. Thus for manual data runs, an 
equivalent manual interarrival time IAE' (defined in 
section B.2) can be obtained. If the aircraft thresh- 
old crossing times, the final approach speed, and the 
wake vortex spacing requirement are known, the IAE' 
can be calculated from 

IAE' = (ATA 2 - ATAi) - (Time to fly scheduled separation) (3) 

Each subject controller’s numerical mean and stan- 
dard deviation of interarrival time error arc given 
in table 4 for the display formats and speed proce- 
dures tested. Also presented are the standard devi- 
ations obtained from lumping (simply combining all 
controller’s data for a condition) and from pooling 
(weighing individual run standard deviations). To 
preserve anonymity, the controller subject numbers 
have been randomized and bear no relationship to the 
original test order appearance; however, listed sub- 
ject numbers are consistent for all report data. The 
histograms resulting from lumping all the subject 
controller interarrival-time errors for a particular dis- 
play format are shown in figure 16. Figure 17 shows 
the excess separation at threshold (actual separation 
distance minus wake vortex required distance) his- 
tograms for the benefit of readers more familiar with 
the distance domain. It should be noted that, for the 
dense arrival traffic condition simulated, figures 16 
and 17 are an approximate time/distance domain 
mapping of each other. 

Figures 16 and 17 give a visual indication of the 
interarrival time or distance error distributions as- 
sociated with each condition tested. However, note 
in table 4 that the mean interarrival time errors 
vary from controller to controller, particularly for 
the manual runs. When the interarrival errors from 
all the controllers are simply combined or lumped 
together, the standard deviation obtained is larger 
than representative of the sampled controllers’ per- 
formance. The overestimation is due to the spread- 
ing effect of the difference among the means. The 
statistical pooled estimate of the standard devia- 
tion (sq) was used as the measure of comparison be- 
cause it overcomes this spreading limitation and more 
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accurately characterizes the spread of the interarrival 
time errors of the final-controller sample tested. The 
pooled estimate of the variance of a population from 
which k samples were taken is defined as 

s 2 _ (ni - 1) sf + (n 2 - 1) 4 + ■ ■ • + ( n fc ~ !) 4 
S ° n\ + ri 2 + ■ ■ ■ + rif; - k 

( 4 ) 

where, respectively, ni, U2, . rik are the number 
of data points in each sample and . .., 


are the calculated variance of each sample around 
its mean. The pooled estimate has n\ + n<i + . . . + 
rifo — k degrees of freedom for making significance 
test or finding confidence intervals. Interarrival time 
error data collected from each controller is treated 
as a sample with data points and sf variance. 
The pooled estimates of the standard deviation of the 
interarrival error are shown in table 4 and are plotted 
in figure 18 with their corresponding 95 percent 
confidence intervals. (Remember that at 210 knots, 
a speed adjustment is made on final.) 


5.1.1* Time Precision of 170-Knot Pattern-Speed Procedure 


Looking at the left side of figure 18 for the results of the 170-knot pattern-speed procedure, we see the 
standard deviations of the interarrival time error are clustered in two groups. The manual/ARTS III (18.9 sec) 
and the centerline slot marker (17.8 sec) formats are near each other and have higher standard deviations than 
the closely grouped pair of the graphic marker (14.7 sec) and DICE countdown (13.9 sec) formats. The results 
are presented of a systematic series of pairwise, one-sided statistical F tests performed to determine whether 
the following variance comparisons (c r 2 > cr 2 ) were significant: 

Manual/FASA statistical comparisons: 

a l 70 MAN > ^170 GM at 0 0005 level of significance (99.95 percent confidence) 

0*170 MAN > ^170 DICE at 0.0005 level of significance (99.95 percent confidence) 

a l70MAN n °t> tTj^ocsM cannot re j cc t null hypothesis (at 0.05 level of significance) 

FASA statistical comparisons: 

a 170CSM > <J 170GM at 0.0005 level of significance (99.95 percent confidence) 

a 170CSM > a i 70 DICE 0.0005 level of significance (99.95 percent confidence) 

^170 GM n °t> ^170 DICE cannot reject null hypothesis (at 0.05 level of significance) 

Statistically a\ was considered larger than a 2 when the level of significance a was less than 0.005 or confidence 
greater than 99.5 percent (i.e., reject null hypothesis that (cr^)’ s are equal when there is up to 0.5 percent risk 
of type 1 error- - up to 0.005 probability that (a 2 )’ s actually were equal). Note that for a series of n pairwise 
tests that are each performed with level of significance a, the experimentwise or joint level of significance a f 
is bounded by cl < 1 — (1 — a) n (ref. 20). Thus, when n = 6 and a = 0.005, the resultant ol was <0.03 
(>97 percent confidence). 

For the 170-knot pattern-speed procedure, statistical analyses of the test results indicate that the centerline 
slot marker format cannot be said to improve the precision of final spacing over that achieved manually with 
no final-approach spacing aid. What this result seems to show is that, when the aircraft and its slot marker 
are traveling at the same 170-knot speed, the judgment of when to turn the aircraft from base-to-final to 
merge with its slot marker is somewhat demanding. In fact, in terms of separation performance, the judgment 
required appears comparable with the judgment (in the manual mode) of when to perform the base-to-final 
turn to separate an aircraft from the preceding aircraft on final. It is worth noting that even though the slot 
marker IAE spread was comparable with that of the manual format, its bias (mean) for most of the subjects 
was less than that of the manual format. This indicates a slightly higher arrival rate (increase from 31 to 32.6 
aircraft per hour based on assumptions of section 5.1.4) could be expected from the centerline slot marker 
because it had less excess separation. 

The F test indicated there was no statistically significant difference between the delivery precision of the 
graphic marker format and the DICE countdown format for the 170-knot pattern-speed procedure. However, 
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we can say, with high confidence (at least 99.7 percent for six pairwise comparisons at a = 0.0005), that both 
the graphic marker and the DICE countdown improved the delivery performance relative to either the centerline 
slot marker or the manual format. The graphic marker and DICE countdown format improved the standard 
deviation of the interarrival time error by 4.2 to 5.0 sec over that achieved manually with no FASA. 

5.1.2. Time Precision of 210-Knot Pattern-Speed Procedure 

The results of the 210-knot pattern-speed procedure are included on the right side of figure 18. The 
standard deviations of the interarrival time error for all the FASA formats are considerably less than for the 
manual/ ARTS III format (15.4 sec). As in the 170-knot procedure, the graphic marker format (9.4 sec) is less 
than the centerline slot marker format (11.2 sec). However, in this case the DICE countdown format (8.2 sec) 
appears to be slightly less than the graphic format . The results are presented of a series of pairwise, one-sided 
statistical F tests performed to determine if the following > cr| comparisons were significant: 

Manual/FASA statistical comparisons: 

^2 10 MAN ^ a 2lOGM a t 0.0005 level of significance (99.95 percent confidence) 

^210 MAN ^ ^210 DICE 0-0005 level of significance (99.95 percent confidence) 

a 2lOMAN ^ ^210 CSM 0.0005 level of significance (99.95 percent confidence) 

FASA statistical comparisons: 

a 2iOCSM ^ a 2lOGM 0.0005 level of significance (99.95 percent confidence) 

^2 10 CSM > ^210 DICE a t 0.0005 level of significance (99.95 percent confidence) 

^210 GM > ^210 DICE 0.005 level of significance (99.5 percent confidence) 

For the 210-knot pattern-speed procedure, we can say with high confidence that the test results for the 
centerline slot marker format show an improvement in precision of final spacing over that achieved manually 
with no final-approach spacing aid. The improvement measured was a reduction of 4.2 sec in the standard 
deviation of the interarrival time error. The graphic marker format, with speed reduction aiding on final, had 
an even better improvement over the manual with a reduction of 6.0 sec. The DICE countdown format with 
speed reduction aiding on final, had the best FASA improvement over the manual format with a reduction 
of 7.2 sec in the standard deviation of the interarrival time error. 


5.1.3. Precision Comparison of 170- and 210-Knot Pattern-Speed Procedures 


The measured reductions in the standard deviation of the interaraval time error for the 210-knot procedure 
were noteworthy. From figure 18, each format’s standard deviation for the 210-knot procedure was reduced 
relative to that for the 170-knot procedure. The results are presented of pairwise, one-sided statistical F tests 
to determine whether the following o\ > comparisons, between the two pattern speeds, were significant: 


2 2 

70 MAN > ^2 10 MAN 

2 2 
^mGM > a 210GM 

2 2 
a \ 70 DICE > a 2 10 DICE 

2 2 

a \ 70 CSM > ^2 10 CSM 

2 2 
^210 MAN not> ^mGM 


at 0.0005 level of significance (99.95 percent confidence) 
at 0.0005 level of significance (99.95 percent confidence) 
at 0.0005 level of significance (99.95 percent confidence) 
at 0.0005 level of significance (99.95 percent confidence) 
cannot reject null hypothesis (at 0.05 level of significance) 


With very high confidence (at least 99.8 percent for four pairwise comparisons at a = 0.0005), we can say 
that the 210-knot pattern-speed procedure, with its speed reduction control, improved the delivery precision 
relative to that of the 170-knot pattern speed for the manual format and each of the FASA formats tested. The 
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reduction in standard deviation for the manual format was 3.5 sec and the reductions for the FASA formats 
were between 5.3 and 6.6 sec. What is particularly noteworthy is that the arrival precision of the 170-knot 
procedure graphic marker format could not be considered different from that of the 210-knot manual format. 
These results indicate that the addition of speed control aiding, on final approach, to a FASA has the potential 
to significantly improve aircraft separation precision at facilities where an initial higher pattern speed on final 
is practical. Recall from section 3.1.5 that slower pattern speeds offer more planning time to organize traffic. 
Another aspect to consider is that when the center and approach elements of TATCA (the Center/TRACON 
Automation System (CTAS)) are operating, higher pattern speeds closer in may be more feasible than in 
today’s environment with no automation decision aids. 


5.1.4. Runway Arrival Rate Benefit 
Assessment 

A simple single runway arrival rate model, con- 
tained in appendix C, was used to give an indica- 
tion of what capacity effect the various measured 
FASA control precisions would have on a time-based 
TATCA system. The conditions assumed are 

Arrival only runway, IMC with constant demand 

Runway occupancy times are less than aircraft 
pair separation times 

Two classes of aircraft — heavy (H) and large (L) 
Traffic is 85 percent large, 15 percent heavy 

Final speed of heavy aircraft past FAF is 140 knots 
GS 

Final speed of large aircraft past FAF is 130 knots 
GS 

5.5 n.mi. from final- approach fix to threshold 

Separation violations restricted to 5 percent 

Arrival rates are plotted in figure 19 for the following 
separation requirements: 



Separation for 


trail aircraft, n.mi. 

Lead aircraft 

Large 

Heavy 

Large 

2.5 

2.5 

Heavy 

5 

4 



Separation for 


trail aircraft, n.mi. 

Lead aircraft 

Large 

Heavy 

Large 

2 

2 

Heavy 

4.5 

3.5 


With the use of the 2. 5/4/5 n.mi. separation 
curve, the arrival rate difference between that of the 
manual format and that of the graphic marker for- 
mat was used to represent the potential benefit of 
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a TATCA system with FASA as compared with a 
TATCA system without FASA. Thus, a FASA theo- 
retically would increase the IMC arrival rate 6.6 per- 
cent for the 170-knot pattern-speed procedure. For 
the 210-knot pattern-speed procedure, there would 
be a potential FASA increase of 10.6 percent. If the 
170-knot manual format is used as the baseline to 
compare against the arrival rate possible by imple- 
menting a higher approach speed on final together 
with turn and speed reduction aiding, the potential 
improvement would be a more dramatic 16.5 percent 
in the IMC landing rate. 

Thinking farther into the future, a more advanced 
integrated TATCA system would use refined weather 
prediction, wake vortex modeling and detection to- 
gether with precise aircraft flair and touchdown, 
high-speed turnoffs, and runway guidance to dynam- 
ically adjust separation times to the minimums pos- 
sible under the atmospheric conditions. When the 
weather would permit the use of time separations 
which transform to 2/3. 5/4. 5 n.mi. distance separa- 
tions, this advanced TATCA system (using the 210- 
knot approach speed with appropriate FASA) could 
theoretically land 44.3 aircraft per hour. This trans- 
lates to a 36.2-percent increase over the baseline man- 
ual format, 170- knot pattern-speed procedure, IMC 
arrival rate. 

5.2. Vectors Per Aircraft in Final Sector 

Each subject controller’s mean number of vectors 
(heading changes) per aircraft in the final sector is 
shown in figure 20(a) for the display formats and 
speed procedures tested. Also shown are the com- 
bined means of all the controller subjects for each 
FASA display format of both pattern-speed proce- 
dures. These combined means are plotted in fig- 
ure 20(b). Note that the mean vectors per air- 
craft were for all aircraft landed during the data 
period. They consisted of both eastern aircraft, on 
base, which required at least one heading change 
to intercept the ILS and western aircraft, on down- 
wind, which normally required at least two heading 
changes. 
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5.2.1. Vectors Per Aircraft for 170-Knot Pattern-Speed Procedure 

The number of vectors per aircraft of the formats for the 170-knot pattern-speed procedure appear to fall 
into three levels — the manual format (2.7 vectors/aircraft) has the highest mean value, the centerline slot 
marker format (2.2 vectors/aircraft) is next, and the DICE countdown (1.6 vectors/aircraft) and the graphic 
marker (1.5 vectors/ aircraft) formats have the lowest number of vectors per aircraft in the final sector. A 
single- factor (display format), repeated measure analysis of variance yielded a P value of 0.0001 for treatment 
differences. The results are presented for Fisher PLSD (protected least significant difference) post hoc, paired 
comparisons to determine whether the differences in measured vectors per aircraft were statistically significant: 

Manual /FAS A statistical comparisons: 

A* 170 MAN > £*170 CSM at 0.005 level of significance (99.5 percent confidence) 

A 4 170 MAN > £*170 DICE a t 0.005 level of significance (99.5 percent confidence) 

Ml 70 MAN > £*170 GM at 0.005 level of significance (99.5 percent confidence) 

FASA statistical comparisons: 

A 4 170 CSM > £*170 DICE a t 0.005 level of significance (99.5 percent confidence) 

Ml 70 CSM > £*170GM a t 0.005 level of significance (99.5 percent confidence) 

£*170 DICE not > /V 170 GM cannot reject null hypothesis (at 0.05 level of significance) 

The analyses indicated that there was no statistically significant difference between the mean vectors per 
aircraft of the graphic marker format and that of the DICE countdown format for the 170-knot pattern-speed 
procedure. However, with high statistical confidence (99.5 percent), we can say both the graphic marker and 
DICE countdown formats reduced the mean number of vectors per aircraft relative to that of both the manual 
and centerline slot marker formats. The measured mean reduction of both the graphic and the DICE relative 
to the manual format was 1.1 vectors/aircraft and their average measured reduction relative to the centerline 
slot marker was 0.6 vector/aircraft. Also with high statistical confidence (99.5 percent), we can say that the 
subjects when operating with the centerline slot marker format, on the average, used fewer vectors per aircraft 
than when operating manually with no FASA. The centerline slot marker measured reduction relative to the 
manual format was an average 0.5 vector /aircraft in the final sector. 

The graphic marker and DICE countdown formats indicated suggested turn location and headings based on 
a classic squared approach pattern (downwind parallel to final and base perpendicular to final). On the other 
hand, the manual technique varied from controller to controller, depending on training and procedures learned 
at current and earlier facilities in their careers. As the individual subject results for the 170-knot procedure 
indicate (fig. 20(a)), a third to a half of the controllers strongly tended to issue intermediate vectors rather 
than a single downwind-to-basc or base-to-final heading. A fair question to raise is whether the manual format, 
vectors per aircraft measured from the test subjects, fairly represents U.S. controllers. In other words, did 
the subject sample have a larger proportion who tended to use intermediate vectors for spacing (as opposed 
to squared turns) than contained in the general controller population? We do not have the data to answer 
the question. However, while more pronounced for subjects 4, 5, 8, 10, and 11, all the subjects had lower 
average vectors per aircraft for their DICE countdown and graphic marker formats than they did for either 
their manual or centerline slot marker formats. 

For the population of controllers tested, the roughly 40-percent reduction in average vectors per aircraft 
observed (between the manual and that of either the graphic or DICE format) has the potential to reduce the 
current communication congestion experienced at major terminals during peak traffic periods. The reduction 
in the number of vectors also appears to have some impact on the distribution of aircraft in the simulation’s 
dump airspace. This relation is complicated somewhat by the fact that imprecision of timing vectors (for 
instance being late on downwind-to-base turn) can also spread aircraft in the dump airspace as well as the 
process of issuing more vectors per aircraft (intermediate vector technique). Figure 21, which plots the aircraft 
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positions every 20 sec, shows the final sector flow pattern for the 170-knot, procedure for all formats (manual, 
centerline slot marker, DICE, and graphic). Clearly the manual format (with the most vectors per aircraft 
and less vectoring precision) has the most widely dispersed (less concentrated) flow pattern of the four. The 
graphic marker and DICE formats (with the least vectors per aircraft and most vectoring precision) have the 
more concentrated dump flow pattern of the formats in figure 21. 

5.2.2. Vectors Per Aircraft for 210-Knot Pattern-Speed Procedure 

As for the 170-knot pattern-speed procedure, the number of vectors per aircraft of the formats for the 
210-knot procedure also appear to fall into three levels (fig. 20(b)). The format order is also the same; however, 
the spread between the extreme values is reduced somewhat. The manual format (2.3 vectors/aircraft) has the 
highest mean value. The centerline slot marker (1.8 vectors/ aircraft) is next. The graphic marker and DICE 
countdown formats (each with 1.5 vectors /aircraft) have the least number of vectors per aircraft in the final 
sector. A single factor (display format), repeated measure analysis of variance yielded a P value of 0.0001 for 
treatment differences. The results are presented for Fisher PLSD (protected least significant difference) post 
hoc, paired comparisons to determine whether the differences in measured vectors per aircraft were statistically 
significant: 

Manual/FASA statistical comparisons: 

M2 10 MAN > M210CSM at 0.005 level of significance (99.5 percent confidence) 

M2 10 MAN > M210DICE at 0.005 level of significance (99.5 percent confidence) 

M210MAN > M210GM at 0.005 level of significance (99.5 percent confidence) 

FASA statistical comparisons: 

M210CSM > M210DICE at 0.01 level of significance (99 percent confidence) 

M210CSM > M210GM at 0.01 level of significance (99 percent confidence) 

M210DICE not> M2 10 GM cannot reject null hypothesis (at 0.05 level of significance) 

The results for the 210-knot pattern-speed procedure parallels those for the 170-knot procedure. No 
statistically significant difference was found in the mean vectors per aircraft for the graphic marker and the 
DICE countdown formats. However, with high statistical confidence (99.5 percent), we can say that the graphic 
marker and DICE countdown formats reduced the mean number of vectors per aircraft relative to the manual 
format and, with high confidence (99 percent), that the same two reduced the number of vectors relative to the 
centerline slot marker format. The measured mean reduction of both the graphic and DICE formats relative 
to manual format was 0.7 vector/aircraft and the mean reduction, of the same two formats, relative to the 
centerline slot marker was 0.3 vector/aircraft. Also with high statistical confidence (99.5 percent), we can say 
that the subjects used fewer vectors per aircraft when operating with the centerline slot marker than when 
operating manually with no FASA. The measured reduction for the centerline slot marker format relative to 
the manual format was 0.5 mean vector/aircraft in the final sector. 

When operating with the 210-knot pattern-speed procedure, the population of controllers tested used about 
30 percent fewer vectors per aircraft for the graphic or DICE formats than they used for the manual format. As 
stated earlier, this has the potential to reduce communication congestion at the major terminals during peak 
traffic periods. As in the 170-knot procedure, there appears to be a correlation between vectors per aircraft and 
the spread of aircraft in the dump airspace, although precision of vector timing also had an effect. Figure 22 
indicates the final sector dump flow pattern for the 210-knot procedure for all formats (manual, centerline slot 
marker, DICE, and graphic). Clearly the manual format (with the most vectors per aircraft and less vectoring 
precision) has the most widely dispersed (less concentrated) flow pattern of the four. The graphic marker 
and DICE countdown formats (with the least vectors per aircraft and most vectoring precision) have the most 
concentrated dump flow pattern of the formats in figure 22. 
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5.2.3. Vectoring Comparison for 170- and 210-Knot Pattern-Speed Procedures 


From figure 20(b), both the manual and the centerline slot marker formats appear to have fewer mean 
vectors per aircraft in the 210-knot procedure than in the 170-knot pattern-speed procedure. A two- factor 
(pattern-speed procedure, display format) repeated measure analysis yielded a P value of 0.0026 for pattern- 
speed effect. The results are presented of paired contrast performed to determine whether the measured 
differences in format, mean vectors per aircraft, between the two pattern-speed procedures were significant: 


Ml 70 MAN > M2 10 MAN 
Ml 70 CSM > M210CSM 
Ml70 DICE not > M2 10 DICE 


Ml 70 GM not > M210GM 


at 0.002 level of significance (99.8 percent confidence) 
at 0.006 level of significance (99.4 percent confidence) 
cannot reject null hypothesis (at 0.05 level of significance) 
cannot reject null hypothesis (at 0.05 level of significance) 


With high confidence (99.8 percent) we can say that the 2 10- knot pattern-speed procedure reduced the 
manual format vectors per aircraft relative to the corresponding 170-knot procedure. With high confidence 
(99.4 percent), we can make the same claim for the 210-knot procedure centerline slot marker format relative 
to the corresponding 170-knot procedure. However, as might be expected, there was no statistically significant 
difference between the 210-knot pattern-speed procedure and that of the 170-knot procedure for cither the 
DICE countdown or graphic marker format. 

No difference between the pattern-speed procedures was expected for the DICE and graphic formats because 
the suggested turn location and heading were based on the same classic squared approach pattern (downwind 
parallel to final and base perpendicular to final) in all cases. However, the fewer vectors in the 210-knot 
procedure than in the 170-knot procedure for both the manual and centerline slot marker formats was not 
expected. Apparently the additional separation fine-tuning control available in the 210-knot pattern-speed 
procedure, from selecting the point of speed reduction, eliminated the need for some separation adjustment 
vectors. For both the manual and centerline slot marker formats, there was an approximate 16-percent reduction 
in the mean vectors per aircraft in going from the 170-knot to the 210-knot pattern-speed procedure. 


5.2.4. FASA Learning Effect on Number of Vectors Issued 


Debriefing discussions and controller comments suggested a learning or training effect after operating with 
the graphic marker and DICE countdown formats. After using the direct automation aids (graphic and DICE 
formats), there seemed to be a tendency among subject controllers toward using more squared downwind/base 
vectors (classic trombone pattern) rather than intermediate vectors (cutting corners) in subsequent manual and 
centerline slot marker runs. Recall from section 4.3, the manual format was always the first format tested in 
both pattern-speed procedure series. The purpose of appendix D is to determine whether the previous vectors 
per aircraft conclusions are voided if this learning effect existed. We asserted that the comparisons and trends 
are correct, and if anything, the “true” mean vectors per aircraft for the manual and centerline slot marker 
formats are somewhat higher than those shown in figure 20(b). 

The result of the appendix D analysis supports the assertion that the subject controllers did change the 
mean number of vectors per aircraft issued, for both the manual and centerline slot marker format, after being 
exposed to the direct automation aid graphic marker and DICE countdown formats. This change was a mean 
reduction of about 0.4 vector/aircraft for the manual format and about 0.3 vector/aircraft for the centerline 
slot marker format. In addition, the analysis reinforced that all earlier mean trend comparisons were valid, 
particularly that the graphic marker and DICE countdown formats had significantly fewer mean vectors per 
aircraft than either the manual or centerline slot marker formats. In fact, the differences are likely larger 
than shown in figure 20(b). Appendix D showed that the mean vectors per aircraft for both the manual and 
centerline slot marker formats in figure 20(b) are likely underestimated by approximately 0.2 vector/aircraft. 
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5.3. Response Time of Subject Controllers to Direct FASA Aids 

The controller response time to the direct aid category FAS As is defined as the difference between the 
ideal message delivery time indicated by the FASA and the actual message delivery time. By definition 
(section 3.1.3), this can only apply to direct’ category FASA’s which have both a specific advisory and desired 
delivery time. Indirect category FASA’s, which only indicate the desired end goal, cannot have a response time 
as defined. With this in mind, response time data were collected on the subject controllers when using the two 
direct category FASA’s tested, the graphic marker and the DICE countdown formats. Data were taken on the 
basc-to-final turn of both the 170- and 210-knot pattern-speed procedures and the nominal 210- to 170-knot 
speed reduction of the 210-knot pattern-speed procedure. 

When considering aircraft-pair separation, the variation or spread in controller response time to a FASA aid 
is much more of interest than the bias. For example, if two aircraft arc both turned late by the same amount 
(i.e., a constant bias in the response time), separation will be preserved if all factors are constant. Figures 23 
through 26 plot the response time data gathered. The actual message delivery time data were obtained from the 
final controller’s coordinator activating a timing button when hearing the controller issue a heading or speed 
reduction command. This process contributed a small amount of imprecision to the measured controller’s 
time of delivery. However, postexperiment test indicated the docker’s (human activating the timer) standard 
deviation was roughly a third or less that of the controller’s response time. Thus, the standard deviations 
measured are a close approximation (only overestimated by about 5 percent) to actual controller performance. 

The response time performance, bctw'ecn the 170-knot and the 210-knot pattern-speed procedures, for the 
graphic marker final-turn advisory, were almost the same. Consequently, results from both speed procedures 
were combined to estimate controller’s response to the turn advisories with the graphic marker format as 
shown in figure 23. The same situation existed for the DICE format case. Therefore, figure 24 represents the 
combined 170-knot and 210-knot procedures response times to the DICE format’s final turn advisory. It should 
be pointed out that the data in figures 23 through 26 are not a straight combination of all controller results. 
Because of the difference in the mean response times among subject controllers, a simple combination of the 
subjects data would result in an unrepresentative picture of the controllers’ response time scatter or standard 
deviation. As a result, each subject controller’s response data were normalized to the mean of the combined 
sample such that deviations from each subject’s mean were plotted as deviations from the combined sample 
mean. 

The results are presented of one-sided statistical F tests performed to determine whether there was statistical 
significance to the differences in measured response time standard deviation between the graphic marker and 
the DICE countdown formats: 

Comparison of graphic/DICE final-turn-advisory response time: 

ff GM > ct DTCE at 0 00003 level of significance (99.997 percent confidence) 

Comparison of graphic/DICE final-specd-reduction- advisory response time: 
a DICE > °gm at 0 -°2 l eve l of significance (98 percent confidence) 

With very high statistical confidence (99.997 percent), we can say that the standard deviation of the 
controllers’ response times to the turn advisories of the DICE countdown format (3.3 sec) was less than that 
to the graphic marker turn advisories (3.7 sec). Recalling the results from section 5.1 as: 

Standard deviation of 170-knot pattern-speed procedure interarrival error: 

Graphic marker 14.7 sec 

DICE countdown — 13.9 sec 
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Standard deviation of 210-knot pattern-speed procedure intcrarrival error: 

Graphic marker 9.4 sec 
DICE countdown 8.2 sec 

The statistically distinct difference in standard deviaion of turn advisory response time, though numerically 
small, appears to account for the slight separation precision advantage measured for the DICE countdown 
format. 

With 98 percent confidence, we can say that the standard deviation of the controllers 5 response times to 
the speed advisories of the graphic marker format (4.1 sec) was less than that to the DICE countdown format 
(4.5 sec). The comparative outcome is reversed from that obtained for the turn advisories. We believe the 
dynamics of the DICE speed advisor countdown was the cause of this reversal. As discussed in sections 3.2.3 
and A. 3, the DICE speed advisor countdown rate changed slowly (approximately 1 count change for every 
4 sec). This pace did not tend to hold the controllers’ attention as did the more rapid DICE turn advisor 
countdown. Thus, the controllers’ response time scatter (standard deviation) for the DICE speed advisor was 
not only greater than that for the DICE turn advisor but also greater than that for the graphic marker speed 
advisor. 


5.4. Evaluation of Lookpoint Data 

The oculometer system basically measures and 
records where the subject is looking; thus it indicates 
what is being looked at and how long any single fixa- 
tion lasts. This doesn’t necessarily identify what the 
controller is thinking. However, eye-tracking data 
are objective and cannot be reliably acquired from 
another type of source, certainly not from interrogat- 
ing the subject himself. By summing fixation times 
over particular objects (such as the aircraft position 
symbols, data blocks), an understanding of the scan 
pattern develops. Likewise, summing over areas of 
the display (zones) demonstrates that the scan is not 
uniformly distributed along the nominal routes. For 
example, figures 27 (manual) and 28 (CSM) are look- 
point scatter diagrams of two of the 96 test runs. 
They each show the position of about 6000 lookpoints 
or fixations. During a 70-min test, the subject is fix- 
ating and being recorded by the oculometer about 
85 percent of the time. Comparing figures 27 and 28 
(lookpoints) with figures 21 and 22, which show air- 
craft positions, the nonuniformity of the scan distri- 
bution along the pattern routes is evident. The final 
controllers looked more frequently along the final- 
approach course and the base legs. Also, notice in 
figure 28 (CSM) the fixations along the extension of 
the final approach where the slot markers appear. 

We examined three measures of lookpoint behav- 
ior as a function of the tested display formats. The 
first measure was the amount of time the oculometer 
had the subject in track which was treated as fol- 
lows: as a percentage of test time, as a percentage 
of time divided among display object types, or as a 


percentage of time divided among regions of the con- 
troller’s display. The second measure was average fix- 
ation time duration by display object type. The third 
measure was number of cross-check scans which indi- 
cated the number of uninterrupted fixations alternat- 
ing between two display objects. Cross-check scans 
are further defined in section E.6. Appendix E also 
contains more information on the oculometer system, 
lookpoint analysis, and related data files. 

In the following sections for each lookpoint mea- 
sure, a single-factor (display format) repeated mea- 
sure analysis of variance was performed for each 
pattern-speed procedure to determine whether the 
display format differences in the particular lookpoint 
measure were statistically meaningful at level of sig- 
nificance of 0.05 (95 percent confidence level). If 
significant, then a Fisher PLSD post hoc, paired 
contrast test was performed to determine which 
paired differences were significant. Also a two- factor 
(pattern-speed procedure, display format) repeated 
measure analysis of variance was performed to deter- 
mine whether there were statistically meaningful dif- 
ferences in the particular lookpoint measure between 
the two speed procedures. The numerical values ob- 
tained from the above test are presented in the final 
section of the lookpoint discussion (sec. 5.4.7). 

5.4.1. Oculometer In- Track Time as 

Percent of Total Test Time 

During the experiments, the controllers had some 
discretion as to when to monitor the display and 
could spend part of the time looking away from the 
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display while talking casually with the feeder con- 
troller or coordinator- For the oculorneter in-track 
time measurement, the working hypothesis relative 
to task difficulty is as follows: the more difficult the 
task, the higher the in-track time is because there 
is less discretionary or spare time to look away from 
the display. The analysis in this section uses this ap- 
parent behavior pattern to differentiate between the 
display formats. 

The data in figure 29 are the time the subject’s 
eye was being tracked by the oculorneter as a per- 
cent of total test time- The figure presents the mean 
of the 12 subjects for each of the four display for- 
mats and for each of the two pattern speeds, a total 
of 96 runs of 70 min each. At the 210-knot pat- 
tern speed, there were no statistically significant dif- 
ferences among the four formats. At the 170-knot 
pattern speed, the graphic format required a statis- 
tically significant, lower percentage of attention or 
viewing time than either the manual or centerline slot 
marker format. The percentage of in-track time for 
the DICE format was between that of the graphic and 
the centerline slot marker and was not statistically 
significantly different from either. This closely agrees 
with the TLX workload results of section 5.5.2. Also 
of interest, the data show that the subjects spent 
about 80 percent of the test runs looking at the radar 
display. Or put another way, the discretionary time 
on the average was about 20 percent. 

The differences in the in-track time as a percent 
of total run time between the 170- and 210-knot 
pattern-speed procedures were not statistically sig- 
nificant at the 0.05 level. The result was that the 
null hypothesis (in-track time as a percent of total 
run time for the two speed procedures arc equal) 
could not be rejected at the 0.05 level of signifi- 
cance. Therefore, the 210-knot pattern-speed pro- 
cedure did not affect in-track time as compared with 
the 170-knot procedure. 

5.4.2. In- Track Time by Display Object 

Type 

A discussion of how much of the total test time 
was spent looking at the radar display was presented 
in section 5.4.1. Now we will address the question 
of how the subject budgets in-track times over the 
various objects on the display and how r , if at all, the 
display formats influence this lookpoint pattern. In- 
track times for various display objects are presented 
in figure 30. Section 5.4.3 addresses how the total 
in-track time is budgeted among display zones, and 
section 5.4.5 examines the average length of individ- 
ual fixations broken down by type of display object 
and test treatment. 


The four principal display object types chosen for 
this analysis are the data block, the aircraft posi- 
tion symbol, the FASA, and the combination of the 
aircraft position symbol and FASA. The first two of 
these exist for all formats. For the 170-knot pro- 
cedure GM runs, the FASA was the turn marker, 
whereas for the 2 10- pat tern- speed procedure, the 
GM FASA also included the speed reduction advi- 
sor. For the CSM tests the FASA was the circle 
which enclosed an aircraft call sign moving along 
the extended runway centerline. For the DICE runs, 
we defined the FASA to be the data block whenever 
the DICE countdown was proceeding whether for a 
turn or a speed change. If the subject looked at the 
data block for some other reason such as to obtain 
the aircraft identification while the countdown was 
proceeding, it was still counted as a fixation on the 
DICE FASA. The reason was that the oculorneter 
position resolution did not allow the determination 
of what the subject is looking at within the data 
block; this introduced an ambiguity and resulted 
in fixations that should have been assigned to the 
data block being assigned to the FASA during DICE 
countdown. A similar problem was caused by the 
close proximity of the data block and aircraft posi- 
tion symbol. On formats other than DICE, errors 
due to assigning fixations to aircraft position sym- 
bols rather than to data blocks and vice versa tend 
to cancel one another. On the DICE format, how- 
ever, because of the position of the countdown in 
the last line of the data block, a bias toward over- 
assigning fixations to the aircraft position symbol oc- 
curred. The fourth principal display object type, the 
combination, only applies to the GM and CSM for- 
mats. For these formats when the aircraft position 
symbol got very close to the FASA (within 0.57 in.), 
we chose to assign the fixation to this combination 
rather than assign the fixation to the closer object of 
the two. Data are presented separately in the tablu- 
lation of figure 30(a), but in the graph (fig. 30(b)) 
the combination data are stacked above either the 
GM or CSM FASA as appropriate. 

From 6 to 8 percent of the time, a display ob- 
ject could not be found within the required 0.57-in. 
proximity to the fixation point. These occurrences 
were tabulated in a no-identification (NO-ID) cate- 
gory, which was assumed to be a measurement de- 
ficiency until looked into further. We had expected 
to find these fixations clustered around some object 
that had been inadvertently omitted from our list of 
possible display objects. However, a scatter plot of 
these points shows a fairly random position distribu- 
tion across the controller’s display. In addition, the 
average length (section 5.4.5) of a given fixation in 
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this category wa is, in a statistical sense, significantly 
shorter than those associated with a known display 
object. Therefore, we conclude that these unattached 
fixations are part of normal scan behavior and are in- 
cluded in the tabulation (fig. 30(a)) for completeness. 
They may indicate fixations while thinking. Other 
display object types (such as lines, navaids, or the 
arrival aircraft sequence list) are not discussed here 
and are not plotted in figure 30(b), but they are in- 
cluded in a category called “other” in the tabulation 
(fig. 30(a)). 

5.4.2. 1. In- Track Time by Display Object for 

170-Knot Pattern-Speed Procedure 

Prom figure 30(b), we see time spent on the data 
block for the 170-knot pattern-speed procedure was 
about equal for the three FASA’s («37 percent) and, 
in a statistical sense, was significantly less than the 
manual format («46 percent). For the aircraft po- 
sition symbol fixations (A/C), the GM and CSM 
were about equivalent (~22 percent) and were, in 
a statistical sense, significantly less than the man- 
ual and DICE which are about equivalent («34 per- 
cent). However, recall the value of the DICE A/C 
position was biased, with the consequences to the 
data as explained subsequently. For fixations on the 
FAS A or FAS A & A/C combination, the GM and 
CSM are about equivalent (~26 percent) and, in a 
statistical sense, both were significantly greater than 
the DICE (^20 percent). However, the DICE FAS A 
value could have been somewhat biased. Notice that 
the majority of time spent on the FAS A for both 
GM and CSM was spent on the combination cate- 
gory, that is, when the aircraft position symbol was 
very close to the FASA. Because of oculometer reso- 
lution limitation described earlier in this section, the 
authors believe that for the DICE format some of 
the A/C position symbol fixations should have been 
shifted to the FASA and some of the FASA shifted 
to the data block. This would have tended to further 
level out the differences between the three FASA’s 
for the data block category, reducing the DICE air- 
craft position percentage; thus, the net effect on the 
DICE FASA percentage was uncertain. 

Even though the measure of the fixation time 
among display objects did not clearly discriminate 
among FASA formats, overall there was a reduction 
of approximately 8 percent in the in-track time spent 
on the data block of the three FASA formats as 
compared with the manual format. Similarly, there 
was a reduction of approximately 13 percent of in- 
track time spent on the isolated aircraft positions; 
this accounted for most of the about 25 percent 
of in-track time spent on the FASA’s. Clearly the 


FASA formats have altered the scan pattern of the 
subjects from the manual format. This was to be 
expected since time spent looking at a FASA must be 
taken from objects normally appearing in the manual 
format. 

5. 4. 2.2. In-Track Time by Display Object for 

210-Knot Pattern-Speed Procedure 

From the graph in figure 30(b), it is apparent 
that the FASA display formats have altered the scan 
pattern of the subjects from the manual format. 
Average percent of in-track time spent on data blocks 
was about equal for the GM and CSM (^33 percent) 
and was, in a statistical sense, significantly less than 
the manual format (^48 percent). Time spent on the 
data block during DICE tests was, in a statistical 
sense, significantly lower (~26 percent) than the 
other three formats, but the measurement system 
tended to bias it low. For the aircraft position 
symbol fixations, the manual (~33 percent) was 
about equivalent to the DICE (^32 percent) and 
both were, in a statistical sense, significantly higher 
than the CSM («23 percent), which was in turn 
significantly higher than the GM («18 percent). 
However, like the 170-knot case, the DICE A/C 
position value was biased too high. For fixations 
on the FASA or FASA & A/C combination, the 
DICE and CSM were about equivalent («29 percent) 
and both less than the GM (~34 percent). Notice 
that the majority of time spent on the FASA for 
both GM and CSM was spent on the combination 
category, that is, when the aircraft position symbol 
is very close to the FASA. As explained earlier, 
due to oculometer resolution limitation, the authors 
believe that for the DICE format some of the aircraft 
position symbol fixations should have been shifted 
to the FASA and some o^the FASA to the data 
block. This shifting would have tended to further 
level out the differences between the three FASA’s 
for the data block category, and reduced the DICE 
aircraft position percentage; thus, the net effect on 
the DICE FASA percentage was uncertain. 

Even though the measure of the fixation time 
among display objects did not clearly discriminate 
among FASA formats, overall there was a reduction 
of approximately 15 percent in the in-track time 
spent on the data block of the three FASA formats as 
compared with the manual format. Similarly, there 
was a reduction of approximately 13 percent in the 
in-track time spent on the isolated aircraft positions. 
This accounted for most of the about 31 percent of in- 
track time spent on the FASA’s. As was true for the 
170-knot procedure, the 210-knot procedure FASA 
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formats have altered the scan pattern of the subjects 
from the manual format. 

5. 4. 2. 3. Comparison of Display Object In-Track 

Time for 170- and 210-Knot Pattern-Speed 

Procedures 

No significant difference was found in the total in- 
track time spent on the aircraft position symbols of 
the 170-knot and the 210-knot procedure. However, 
the pattern speed was a significant factor for the data 
block (a = 0.01), for the FASA (a = 0.001), and for 
the aircraft position symbol and FASA combination 
(a = 0.02). For the higher pattern speed, more time 
was shifted to the FASA’s from the aircraft position 
symbols and data blocks. For the DICE and GM, 
the simple explanation was that the added speed 
advisory means that the FASA was encountered one 
more time per arrival. For the CSM, the effect was 
less pronounced and came from using the aircraft’s 
proximity to the slot marker as a cue to reduce to 
the lower pattern speed (170 knots). 

5.4.3. In- Track Time by Display Zones 

In this section we examine how the subjects bud- 
get their fixation time over the entire area of the dis- 
play. In section 5.4.1, the subjects were fixated on the 
display on an average of about 80 percent of the test 
time. In section 5.4.2, this time was divided among 
the specific display object types. The focus here is 
on where the subject is looking rather than at what. 
The final controller’s PPI display is illustrated in fig- 
ures 12 and 13(b). For analysis purposes the final 
sector display was divided into four zones: 

1. Final- approach leg 

2. Base legs 

3. Downwind legs 

4 . Everything else 

Figure 31 is a graphical representation of these zones. 
Note that the zones are not strict areas but are 
defined by a combination of the TIMER approach 
phase status (section 4.1.2) and the heading. For 
instance, the base leg was defined to be within a 
range of headings relative to the nominal base leg 
(perpendicular to final course) and could be at a 
different location for each aircraft. 

Figures 21 and 22 showed that the aircraft posi- 
tion symbols were distributed over the display more 
or less uniformly along the nominal pattern paths. 
However, as discussed earlier, the scatter plots of 


fixations (figs. 27 and 28) indicated the controller 
fixations were not correspondingly uniformly distrib- 
uted. Data of in-track time by display zone verified 
those observations. Only about 4 percent of the con- 
troller’s time is spent in zone 4, whereas between 76 
and 82 percent of their time is spent in zone 1 or 2. 
This section is concerned with describing the scan be- 
havior of the subjects referenced to zones and show- 
ing how, if at all, the various display formats influ- 
ence this behavior. Figure 32 is a plot of controller 
means for the relative amount of in-track time spent 
in each zone for the four display formats and both 
pattern speeds. 

5.4.3. 1. In-Track Time by Zone for 170-Knot 

Pattern-Speed Procedure 

From figure 32, significantly (as used in this sec- 
tion, in a statistical sense) more time was spent 
observing the final approach during the CSM tests 
(~57 percent) than was spent observing this area 
for the other three format tests. The manual for- 
mat (~51 percent) was the second highest and it 
was significantly higher than GM and DICE which 
were about equal (?^44 percent). For the base legs, 
the situation was reversed with GM and DICE about 
equal (%33 percent) and significantly higher than the 
manual and the CSM (^26 percent). For the down- 
wind legs, GM and DICE were again about equal 
(~ 13 percent) and significantly higher than the man- 
ual (^10 percent), which was in turn significantly 
higher than CSM (^8 percent). For the CSM, the fo- 
cus on the spatial relation between the aircraft posi- 
tion marker arid the slot marker increased the obser- 
vation intensity along the final, whereas for the GM 
and DICE, the focus was on the base legs or down- 
wind legs where the FASA’s were displayed. This 
same effect could also be seen for the 210-knot pat- 
tern speed. For the manual format, the controller 
was concentrating on separation and speed along the 
final-approach course. The subject tended to mon- 
itor the data block for aircraft decelerations, con- 
cerned that the aircraft will close on the one ahead. 
When using the FASA’s, especially the DICE and 
GM, the controller tended to use the FASA to turn 
the aircraft rather than using the relative position of 
the aircraft ahead on the final. This favored zones 2 
and 3 over zone 1. With the FASA, some controllers 
assumed that separation and speed on the final ap- 
proach would be taken care of without intervention. 
During CSM runs, the controllers were encouraged 
to make the turn to base leg from the dowmwind leg 
based on preceding traffic rather than keying on the 
slot marker. 
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5.4.3. 2. In- Track Time by Zone for 210-Knot 
Pattern-Speed Procedure 

From figure 32, qualitatively the 210-knot data 
look very much like the 170-knot data. Significantly 
(in a statistical sense) more time (»61 percent) 
was spent on the final approach during the CSM 
tests than during the other three format tests. The 
manual format was the second highest (~57 percent) 
and was significantly higher than GM and DICE, 
which were about equal (^50 percent). For the base 
legs, the situation was reversed with the GM and 
DICE about equal («28 percent) and significantly 
higher than the manual and the CSM (~21 percent). 
On the downwind legs, GM and DICE were again 
about equal (^11 percent) and significantly higher 
than the manual and CSM («8 percent). For the 
CSM, there was an increase in observation around 
the turn onto the final approach, whereas for the 
GM and DICE, activity had shifted to the base legs 
and downwind legs. The effect was stronger in the 
210-knot procedure than in the 170-knot procedure. 

5. 4. 3. 3. Zone In-Track Time Comparison of 170- 

and 210-Knot Pattern-Speed Procedures 

The pattern speed is a significant factor in zone 1 
(a = 0.001), zone 2 (a = 0.0002), and in zone 3 
(a = 0.02). The 170- and 210-knot pattern speeds 
appear similar qualitatively, but there is a definite 
shift toward the final approach and away from the 
base legs and downwind legs with the 210-knot pro- 
cedure. The 210-knot procedure speed reduction on 
final seems to account for the attention shift. 

5.4.4. In-Track Time Inside 

Final-Approach Fix 

The final-approach fix (FAF) is shown in figures 1 
and 31. The amount of controller monitoring inside 
the FAF is critical because of both the separation 
compression that normally occurs in the vicinity of 
the outer marker as well as the differences in individ- 
ual aircraft speeds on final approach. The concern 
was that aircraft separation monitoring inside the 
FAF might decrease as a result of the automation. 
Figure 33(a) gives the subject controllers’ time spent 
inside the FAF, as percentages of their in-track time, 
for the various test conditions. Figure 33(b) shows 
the mean for all subjects combined. The findings in- 
dicate a significant reduction in the monitoring with 
the centerline slot marker format; this is particularly 
relevant when one recalls that the CSM had the worst 
delivery precision of the FASA’s (section 5.1). From 
controller comments (section 5.5.1) and workload 


analysis (section 5.5.2), the CSM reduction in mon- 
itoring relative to the GM and the DICE seems to 
be because of the difficulty and workload of initially 
separating aircraft on final with CSM. 

The 12 subjects spent significantly less time 
(fig. 33) looking at aircraft positions or data blocks 
inside the final-approach fix during the CSM sessions 
than during any of the other three formats. For the 
170-knot pattern-speed procedure, the percentage of 
in-track time for the manual, DICE, and GM were 
about equivalent (7 percent) and were significantly 
higher than the CSM (3.5 percent). For the 210-knot 
pattern-speed procedure, the manual tests had the 
highest percentage (11.2 percent) of time spent in 
this area of the display. The percentage of in-track 
time for the DICE and GM were about equivalent 
(8 percent) and were significantly lower than the 
manual and higher than the CSM (4 percent). 

As a group, the 210-knot pattern-speed procedure 
did not affect the percentages of in-track time spent 
inside the FAF as compared with the 170-knot pro- 
cedure. However, as can be seen from a comparison 
of the manual format at the two speeds in figure 33, 
a notable difference occurs between those two means. 
A likely explanation for the significantly different fix- 
ation time spent inside the final- approach fix for the 
210-knot manual case was that additional time was 
devoted to monitoring aircraft pairs when the fol- 
lowing aircraft was issued a speed reduction. Partic- 
ularly close scrutiny was given to this situation while 
the following aircraft was decelerating, during which 
time, in many instances, the lead aircraft had passed 
the final-approach fix. 

5.4.5. Mean Dwell Times by Display 

Object Type 

Longer average dwell times can be indicative of 
either more information being transferred or, for the 
same amount of information, a slower information 
transfer process. For the results presented in this 
section, the mean dwell time for a display object 
type (for a given subject) was averaged over the test. 
The composite controller means are broken down 
by display object type, FAS A display format, and 
pattern speed and are presented in figure 34. 

A previous section (5.4.2) presented a discussion 
of the total in-track time by display object types 
and of some biases in the data collection process. 
The assertion was that because of the bias, some 
of the DICE FASA fixations probably belonged to 
the data block category which would tend to even 
out the effect of the FASA, that is, to increase the 
shorter amount of in-track time spent on the normal 
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data block and decrease the longer amount of in-track 
time spent on the DICE countdown aid. However, in 
the present section, the same bias has the opposite 
effect. Removing the shorter, misallocated, dwell 
times from the computation of the DICE FASA mean 
would make the mean dwell time (while the FASA 
was active) even longer than shown, that is, would 
amplify further the effects of the DICE FASA. 

5. 4. 5.1. Mean Dwell Time by Display Object for 

170-Knot Pattern-Speed Procedure 

The mean dwell times for the data block were 
about equal for the manual and GM (0.7 sec), slightly 
higher for the CSM (0.8 sec), and barely lower for 
the DICE (0.65 sec). For the A/C position symbol 
fixations the manual, GM, and DICE were about 
equivalent (0.7 sec) and were significantly less than 
the CSM (0.8 sec). For fixations on the FASA, 
the GM and CSM were about equivalent (0.7 sec) 
and both were significantly shorter than the DICE 
(1.3 sec). For the combination category (when the 
A/C position symbol is very close to the FASA), the 
GM and CSM were about equivalent (1.0 sec). The 
most prominent feature of figure 34(b) is that the 
longest dwell times were associated with the FASA 
in the DICE format when the countdown is active 
even though the estimates presented are conservative 
due to the bias already discussed. The next longest 
dwell times were associated with the combination 
of aircraft position symbol and FASA for the GM 
and CSM which occur when the FASA and A/C are 
in close proximity at turn points. Dwell times on 
the data block were very long when the DICE was 
active. They were longer than dwell times for the 
graphic marker or slot marker taken by themselves or 
when combined with the aircraft in close proximity. 
Since the mean DICE dwell time was computed for 
fixations close to the time to turn as well as earlier, 
perhaps a better measure would be to compare the 
DICE FASA average with the weighted average of the 
two object categories, FASA and FASA & A/C for 
both graphic and slot marker. This measure resulted 
in the 1.26 sec for the DICE being compared with 
0.92 sec for the GM and 0.80 sec for the CSM. Clearly 
the DICE mean dwell times are considerably longer. 

Another general point of interest was the active, 
constantly darting nature of the controller’s scan, 
which was seen on video tapes of the controller’s dis- 
play with the lookpoint electronically superimposed. 
Even when a turn was nearing, the controller’s look- 
point did not stay focused on the FASA but darted 
to other traffic and then returned to the FASA only 
slightly before the indicated turn issue time. For 


example, the graphic marker FASA & A/C combi- 
nation mean dwell time was only about 1 sec even 
though that category denotes an impending turn. 
This means that on the average, the lookpoint re- 
mained focused on the graphic turn FASA & A/C 
combination for no longer than a second and then 
darted to another object. This constant motion 
scanning behavior was observed for all formats and 
pattern-speed procedures. 

5. 4. 5. 2. Mean Dwell Time by Display Object for 

210-Knot Pattern-Speed Procedure 

The mean dwell times for the data block was 
about equal for the manual, GM, and DICE (0.7 sec) 
and slightly higher for the CSM case (0.8 sec). For 
the aircraft position symbol fixations, the manual 
and GM were about equivalent (0.7 sec) and were 
significantly less than the DICE (0.8 sec) and CSM 
(0.9 sec). For fixations on the FASA, the GM and 
CSM were about equivalent (0.7 sec) and both were 
significantly shorter than the DICE (1.3 sec). For 
the combination category (when the A/C position 
symbol is very close to the aid), the GM (1.0 sec) 
was significantly lower than the CSM (1.3 sec). 

Dwell times on the data block when the DICE 
was active were very long. They were longer than 
dwell times for the graphic marker or slot marker 
taken by themselves and longer than the FASA & 
A/C combination for the graphic marker. They were 
about equivalent for the FASA & A/C combination 
for the slot marker. However, by using the approach 
discussed for the 170-knot procedure, when the DICE 
was compared with the weighted average of the two 
object categories, FASA and FASA & A/C, for both 
graphic and slot marker, this measure resulted in the 
1.26 sec for the DICE as compared with 0.85 sec for 
the graphic marker and 0.94 sec for the slot. The 
DICE format clearly stands out as having the longest 
average dwell times. 

5. 4. 5. 3. Comparison of Mean Dwell Time by 

Display Object for 170- and 210-Knot 

Pattern-Speed Procedures 

The pattern speed was a significant factor for data 
block ( a = 0.02), aircraft position symbol (a = 0.01), 
and for the FASA and aircraft position symbol com- 
bination (a = 0.02). These effects were mainly due 
to the increase in fixation times during the CSM 
tests. The greatest difference between the two pat- 
tern speeds was in the FASA and aircraft position 
symbol combination for the CSM case. This very sig- 
nificant increase (from 1 to 1.3 sec) at the 210-knot 
speed was probably due to the subject using the slot 
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marker to help make a judgment on when to slow the 
aircraft. Notice that at both speeds the NO-ID phe- 
nomenon, as discussed in section 5.4.2, stands out for 
having short mean dwell times. Also, at either speed, 
when the aircraft got close to its FAS A, there was a 
significant increase in mean dwell time for both the 
GM and CSM. 

5. 4. 5. 4. Comparison of DICE and Graphic 

Marker Mean Dwell Time 

Instrument bias errors discussed earlier tend to re- 
duce the measured DICE mean dwell time (true value 
longer than plotted). Another factor which had the 
opposite effect was the controller’s practice of looking 
at an aircraft call sign just before issuing control in- 
structions. After the final focus on the graphic turn- 
FAS A/aircraft combination before issuing the turn, 
it was normal for the controller to look up to the 
data block for the aircraft call sign. Because of in- 
strument resolution limitation, when a similar scan 
sequence occurred with the DICE format, the time 
spent on the aircraft call sign in the data block was 
credited as part of the DICE dwell time. Thus, the 
measured dwell time for the final DICE focus only 
during each countdown was inflated. The opposing 
bias factors created a slight uncertainty in the value 
of the DICE mean dwell time; thus, relative to the 
simple hypothesis, in a strict statistical sense, we can- 
not make quite as strong a conclusion as that so very 
strongly indicated by the average dwell time data 
values. 

The starting hypothesis about dwell time was 
that longer mean dwell times are indicative of either 
more information being transferred or, for the same 
amount of information, a slower information transfer 
process. The graphic marker and DICE appear 
to provide equivalent information but in different 
form. If the hypothesis is accepted, the fact that the 
mean dwell times were longer for fixations using the 
DICE countdown than the graphic marker suggests 
a more efficient information transfer process for the 
graphic marker format. Intuitively, there seems to be 
some truth to that conclusion particularly because 
controllers are accustomed to using position data 
on their display to make decisions. It is tempting 
to reason that a pictorial representation is easier 
to assimilate than digital information depicting the 
same situation. At a glance the graphic marker 
indicates distance remaining before a turn, whereas 
with a countdown (particularly a DICE countdown) 
there could be a tendency to linger momentarily 
to get a sense of the countdown rate for the same 
information. 


5.4.6. Cross-Check Scans by Zone Pairs 

Cross-check scans (CCS) indicate the number of 
uninterrupted dwell points alternating between two 
display objects and are also discussed in section E.6. 
For a pair of display objects A and B, the lookpoint 
sequence A-B is counted as a CCS of order 2, the 
sequence A-B-A is counted as a CCS of order 3, 
A-B- A-B is of order 4, and so forth. For the zones 
defined in section 5.4.3, a cross-check scan between 
two zones is simply a cross-check between two objects 
located in each of the zones. For analysis purposes 
three redundant groups of zone pairs were selected 
to emphasize the role of the three major zones: final- 
approach leg, base legs, and downwind legs. The 
zone pairs are nondirectional (i.e., Z1/Z2 = Z2/Z1). 
Each group contains the sum of three zone pairs as 
follows: 

Zl/All = Zl/Zl + Z1/Z2 + Z1/Z3 
Z2/A11 = Z2/Z1 + Z2/Z2 + Z2/Z3 
Z3/A11 = Z3/Z1 + Z3/Z2 + Z3/Z3 (5) 

Figures 35(a) and (b) contain the number of cross- 
check scans, of order four or greater, per run for the 
three groups of zone pairs, broken down by subject 
controller and display format. Figure 35(c) graphs 
the mean cross-scans for the 12 subjects. Each point 
on the plot represents the average of 12 70-min trials 
for the indicated pattern speed and display format. 

For the controller, cross-check scans consist of se- 
quentially examining relative positions of aircraft to 
other aircraft, as well as aircraft to geographical (or 
other significant) points on the display. The nor- 
mal purpose of cross-checking is to either perform 
some control action or to monitor separation. The 
hypothesis was that a reduction in the number of 
cross-checks primarily indicated a reduction in the 
amount of comparison or judgment required to prop- 
erly time a control action if the amount of monitor- 
ing is assumed to be relatively constant. According 
to this hypothesis, the results indicate a significant 
graphic marker advantage relative to the amount of 
position comparisons performed. 

Most of the cross-check scanning involves check- 
ing positions along the final- approach course or, to 
a lesser degree, along the base legs, this agrees with 
the other eye scanning measurements and seems to 
indicate the priorities of controllers. On the down- 
wind legs at both speeds, the average CCS’s for GM 
and CSM look about equal (13) and less than for the 
manual and DICE (19). However, because there are 
so few in this group, we hesitate to draw any firm 
conclusions. 
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5.4.6. 1. Cross-Check Scans by Zone Pairs for 

170-Knot Pattern-Speed Procedure 

At the 170-knot pattern-speed procedure on both 
the final- approach and base legs (zones 1 and 2), the 
graphic marker has significantly lower average CCS’s 
than do the other three formats (fig. 35(c)). Also, the 
DICE and CSM are about equal and are significantly 
less than the manual. Therefore, the data indicate 
that on the critical base legs and final-approach leg, 
the manual is associated with the highest average 
CCS’s, the graphic with the lowest average and the 
other two formats are in between. 

5. 4. 6. 2. Cross-Check Scans by Zone Pairs for 

210-Knot Pattern-Speed Procedure 

At the 210-knot pattern-speed procedure on the 
final approach, the graphic marker has significantly 
lower mean CCS’s than do the other three formats 
(fig. 35(c)). On the base legs, the graphic and 
CSM are about equal and significantly less than the 
manual and DICE. Therefore, the data indicate that 
on the base legs and final-approach leg, the manual 
is associated with the highest average CCS’s, the 
graphic with the lowest mean and the other two 
formats are in between. 

5. 4. 6. 3. Comparison of Cross-Check Scans by 

Zone Pair for 170- and 210-Knot Pattern-Speed 

Procedures 

The pattern speed is a significant factor (a = 0.01) 
only in zone 1 (Zl/All). That is, a greater number 
of CCS’s occurred at the higher pattern speed on 
the final approach. This result is consistent with 
the added requirement of slowing the aircraft to 
170 knots on the final-approach course. 

5. 4. 6. 4. Comparison of DICE and Graphic 

Marker Cross-Check Scans 

Clearly the graphic marker has less CCS’s than 
the other formats particularly in the Zl/All and 


Z2/A11 comparisons. As was stated in section 5. 4. 5. 4, 
the graphic marker and the DICE appears to provide 
equivalent information but in different forms. Why 
the observed differences in CCS’s? When the GM 
appears on the display it is immediately apparent 
in the subject’s parafoveal vision so that the subject 
does not have to scan for the occurrence of the event. 
However, w T ith the DICE when an aircraft is on the 
downwind leg, the parafoveal vision does not clue 
the subject that a turn is imminent. Consequently, 
subject controllers tended to use one of two distinct 
eye-scanning patterns with the DICE FASA. 

One scan pattern compares the position of the 
aircraft on the dowmwind leg and final-approach leg 
to estimate when the turn should be issued and 
then the subject starts to monitor the countdown 
more closely. This DICE scan pattern resembles a 
traditional manual scan pattern and tends to have 
more aircraft cross-checking relative to the graphic. 
The fact that the DICE is nonlinear on the downwind 
amplifies this effect. With the GM, the subject has a 
better feel for time remaining. The subject can pick 
up this information without as much cross-checking. 
The other scanning pattern for the DICE seems to 
consist of scanning the data block searching for the 
existence of a DICE countdowm. The second pattern 
results in more data block cross-checking relative to 
DICE. 

Since both the DICE eye-scan patterns described 
tend to increase DICE CCS’s relative to GM, the net 
result is that there is more overall cross-checking in 
the DICE format than in the GM format. Does use 
of the first DICE scan pattern result in maintaining 
a better overall traffic picture than use of the sec- 
ond DICE scan pattern? Does the graphic format 
result in a better traffic picture than cither of the 
DICE scan patterns? Our data provided no answer 
to these two interesting questions or to the more gen- 
eral question of whether reliance on a FASA affects 
controller traffic awareness. 


5.4.7. Summary of Lookpoint Data Statistical Test 

This section presents the F-test statistic (F) and the resultant level of significance (P) values from a single- 
factor (display format) repeated-measure analysis of variance (ANOVA). The ANOVA was performed, for each 
pattern- speed procedure (first table of each measure), to determine whether the display format differences in 
the lookpoint measures were statistically meaningful at a level of significance of 0.05 (95 percent confidence 
level). If the ANOVA for a pattern speed indicated significance ( P < 0.05), a Fisher PLSD post hoc, paired 
contrast test (second table or tables of each measure) was then performed for that pattern speed to determine 
which paired differences were significant (P < 0.05). In addition, a tw r o-factor (pattern-speed display format) 
repeated- measure ANOVA w^as performed to determine whether there w T ere statistically meaningful differences 
in the particular lookpoint measure between the two pattern-speed procedures. These statistical test data arc 
presented in the following tables. 
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In-Track Time (section 5.4.1 and fig. 29): 




ANOVA results for- 


Format factor 
comparison for 
170 knots 

Format factor 
comparison for 
210 knots 

Format and speed 
factors comparison for 
170 & 210 knots 

F 

3.395 

0.413 

2.171 

P 

0.029 

0.744 

0.169 


Format 

P of Fisher PLSD post hoc for 
170-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.008 

0.090 

0.744 

GM 


0.296 

0.018 

DICE 



0.165 


In-Track Time Spent on Data Block (section 5.4.2 and fig. 30): 



ANOVA results for 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

10.900 

24.156 

9.838 

P 

i 

0.000 

0.000 

0.010 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.000 

0.000 

GM 


0.016 

0.795 

DICE 



0.008 


Format 

P of Fisher PLSD post hoc for 
170-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.000 

0.000 

GM 


0.106 

0.968 

DICE 



0.098 


In-Track Time Spent on Aircraft Positions (section 5.4.2 and fig. 30): 



ANOVA results for 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

18.287 

16.942 

1.683 

P 

0.000 

0.000 

0.221 


Format 

P of Fisher PLSD post hoc for 
170-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.136 

0.000 

GM 


0.000 

0.870 

DICE 



0.000 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.535 

0.000 

GM 


0.000 

0.045 

DICE 



0.002 
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In- Track Time Spent on FAS A (section 5.4.2 and fig. 30): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

31.175 

59.569 

61.095 

P 

0.000 

0.000 

0.000 



P of Fisher PLSD post hoc for 


170-knot format paired contrast 

Format 

DICE 

CSM 

GM 

0.000 

0.221 

DICE 


0.000 



P of Fisher PLSD post hoc for 


210-knot format paired contrast 

Format 

DICE 

CSM 

GM 

0.000 

0.016 

DICE 


0.000 


In- Track Time Spent on FASA & A/C Combination (section 5.4.2 and fig. 30): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

0.291 

0.008 

8.809 

P 

0.600 

0.930 

0.013 


In- Track Time Spent in Zone 1 (Final- Approach-Course Leg) (section 5.4.3 and fig. 32): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

63.019 

24.472 

21.832 

P 

0.000 

0.000 

0.001 


Format 

P of Fisher PLSD post hoc for 
170-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.001 

0.000 

0.002 

GM 


0.526 

0.000 

DICE 



0.000 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.000 

0.037 

GM 


0.199 

0.000 

DICE 



0.000 
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In- Track Time Spent in Zone 2 (Base Leg) (section 5.4,3 and fig. 32): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

13.003 

12.067 

29.715 

P 

0.000 

0.000 

0.000 



P of Fisher PLSD post hoc for 


170-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.001 

0.000 

0.269 

GM 


0.773 

0.000 

DICE 



0.000 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.000 

0.907 

GM 


0.919 

0.000 

DICE 



0.000 


In- Track Time Spent in Zone 3 (Downwind Leg) (section 5.4.3 and fig. 32): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

14.260 

15.195 

7.554 

P 

0.000 

0.000 

0.019 



P of Fisher PLSD post hoc for 
170-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.015 

0.001 

0.020 

GM 


0.367 

0.000 

DICE 



0.000 



P of Fisher PLSD post hoc for 
210-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.000 

0.001 

0.351 ~ 

GM 


0.325 

0.000 

DICE 



0.000 


In- Track Time Inside FAF (section 5.4.4 and fig. 33): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 k 210 knots 

F 

4.900 

18.061 

2.724 

P 

0.006 

0.000 

0.127 



P of Fisher PLSD post hoc for 
170-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.453 

0.800 

0.010 

GM 


0.618 

0.001 

DICE 



0.005 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.013 

0.002 

0.000 

GM 


0.451 

0.000 

DICE 



0.000 
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Mean Dwell Time of Fixations on Data Block (section 5.4.5 and fig. 34): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

12.877 

5.511 

7.666 

P 

0.000 

0.004 

0.018 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.468 

0.833 

0.002 

GM 


0.350 

0.012 

DICE 



0.001 


Format 

P of Fisher PLSD post hoc for 
170-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.182 

0.000 

0.098 

GM 


0.006 

0.004 

DICE 



0.000 


Mean Dwell Time of Fixations on Aircraft Positions (section 5.4.5 and fig. 34): 




ANOVA results for — 


Format factor 
comparison for 
170 knots 

Format factor 
comparison for 
210 knots 

Format and speed 
factors comparison for 
170 & 210 knots 

F 

10.088 

14.165 

10.895 

p\ 

0.000 

0.000 

0.007 



P of Fisher PLSD post hoc for 


210-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.649 

0.003 

0.000 

GM 


0.001 

0.000 

DICE 



0.071 



P of Fisher PLSD post hoc for 


170-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.065 

0,883 

0.001 

GM 


0.048 

0.000 

DICE 



0.002 


Mean Dwell Time of Fixations on FASA (section 5.4.5 and fig. 34): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

23.946 

39.751 

0.067 

P 

0.000 

0.000 

0.800 



P of Fisher PLSD post hoc for 


210-knot format paired contrast 

Format 

DICE 

CSM 

GM 

0.000 

0.063 

DICE 


0.000 



P of Fisher PLSD post hoc for 


170-knot format paired contrast 

Format 

DICE 

CSM 

GM 

0.000 

0.047 

DICE 


0.000 
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Mean Dwell Time of Fixations on FAS A & A/C Combinations (section 5.4.5 and fig. 34): 



ANOVA results for 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

0.753 

23.035 

8.912 

P 

0.404 

0.001 

0.012 


Format 

P of Fisher PLSD post hoc for 
2 10- knot format paired contrast 

CSM 

GM 

0.001 


Mean Cross-Check Scans for Z1 (Final)/All Zone-Pair Combinations (section 5.4.6 and fig. 35): 



ANOVA results for — 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 Sc 210 knots 

F 

9.679 

22.297 

11.093 

P 

0.000 

0.000 

0.007 


Format 

P of Fisher PLSD post hoc for 
170-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.007 

0.018 

GM 


0.019 

0.007 

DICE 



0.684 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.000 

0.000 

GM 


0.000 

0.004 

DICE 



0.312 


Mean Cross-Check Scans for Z2 (Base)/All Zone-Pair Combinations (section 5.4.6 and fig. 35): 



ANOVA results for— 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 Sc 210 knots 

F 

8.433 

13.248 

1.372 

P 

0.000 

0.000 

0.266 



P of Fisher PLSD post hoc for 
170-knot format paired contrast 

Format 

GM 

DICE 

CSM 

MAN 

0.000 

0.044 

0.007 

GM 


0.007 

0.047 

DICE 



0.427 


Format 

P of Fisher PLSD post hoc for 
210-knot format paired contrast 

GM 

DICE 

CSM 

MAN 

0.000 

0.317 

0.000 

GM 


0.000 

0.139 

DICE 



0.005 
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Mean Cross-Check Scans for Z3 (Downwind) /All Zone-Pair Combinations (section 5.4.6 and fig. 35): 



ANOVA results for - 


Format factor 

Format factor 

Format and speed 


comparison for 

comparison for 

factors comparison for 


170 knots 

210 knots 

170 & 210 knots 

F 

3.742 

5.829 

0.001 

P 

0.020 

0.003 

0.979 



P of Fisher PLSD post hoc for 
170-knot format paired contrast 



P of Fisher PLSD post hoc for 
210-knot format paired contrast 

Format 

GM 

DICE 

CSM 


Format 

GM 

DICE 

CSM 

MAN 

0.021 

0.699 

0.079 


MAN 

0.064 

0.080 

0.120 

GM 


0.008 

0.544 


GM 


0.001 

0.752 

DICE 



0.035 


DICE 



0.002 


5.5. Controller Questionnaires and 
Verbal Debriefing 

The subjective data were collected in the form of 
questionnaires and a verbal debriefing conducted at 
the conclusion of the data runs. The types of ques- 
tionnaires and the time and order of their adminis- 
tration to a test-subject controller, were as follows. 

At the completion of each data run: 

Format questionnaire (section 5.5.1) 

Task Load Index (TLX) format rating relative to 
six workload factors (section 5.5.2) 

At completion of all four format data runs for a given 
pattern-speed procedure: 

Rating and ordering of the four test display for- 
mats for a speed procedure (section 5.5.3) 

At completion of all data runs: 

Combined rating and ordering of the test display 
formats of both speed procedures (section 5.5.3) 

Pair-wise comparison of the six TLX source-of- 
workload factors (section 5.5.2) 

Comparison of 170- and 210-knot turn-to-final 
procedures (section 5.5.4) 

Final debriefing questionnaire (section 5.5.5) 

Verbal final debriefing (section 5.5.5) 

5.5.1. Format Questionnaires 

The format questionnaires administered at the 
conclusion of each data run consisted of questions 


regarding the format just tested along with some 
general questions about the simulation. These ques- 
tionnaires, including the subject responses, are pro- 
vided in section F.2. The present section provides a 
summary of the format questionnaire results. 

5.5.1. 1. Questions Common to Two or More 

FASA Formats 

Subjects were asked whether they believed that 
the FASA’s created too much clutter. Most re- 
sponded that, although there is inherent clutter 
brought on by displaying additional information, 
clutter was not a problem. One subject did spec- 
ulate that with a “busier” video map, it could be a 
problem. (Fig. 13(b) depicts the video map used by 
the final controller in the study.) As to the time when 
the DICE countdown or graphic marker appeared on 
the display, the subject controllers overwhelmingly 
felt that there was neither too much nor too little 
warning prior to the time the command should be 
issued. Subjects also generally agreed with the au- 
tomation’s choice of where or when to issue turns and 
speed reductions. The sequence list that appeared on 
the right side of the controllers display for all runs us- 
ing display FASA’s was generally felt by the subjects 
to be of little use because of the well-spaced and or- 
dered traffic flow which resulted in an obvious landing 
sequence. The vast majority of subjects strongly fa- 
vored the use of FASA’s in response to the question 
regarding their reaction toward having a computer 
suggest when or where to turn an aircraft. This re- 
action is also supported by comments received in the 
final debriefing sessions. 
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5. 5. 1.2. Manual Format Questionnaire 

5. 5. 1.2.1. 170- knot procedure 

As described in section 3.1.2, the simulated traffic 
flow into the final sector had the properties of having 
been organized and spaced by automat ion/ feeder- 
controller interaction. Based on this organization 
and spacing, all subjects strongly felt that the land- 
ing sequence was apparent. In terms of the effort 
required to “set up” the landing sequence, the con- 
sensus (10 subjects) was that the automation was 
helpful as compared with an operation in which 
no automation existed to meter the traffic and re- 
solve ties. The simulated radar display and the ini- 
tial briefing were considered fully adequate by 10 
and 9 subjects, respectively. The remaining subjects 
noted minor deficiencies; however, none of the sub- 
jects felt that the simulated radar display nor the 
briefing was inadequate. Several questions addressed 
the realism of different simulation components, for 
example, aircraft fight paths and maneuvers, com- 
munications with pseudopilots, and interaction with 
the feeder controller. These elements of the simula- 
tion were evaluated by most subjects to be somewhat 
realistic to realistic. 

5.5. 1.2.2. 2 10- knot procedure 

As in the 170-knot procedure, it was strongly 
felt by all subjects, that the landing sequence was 
apparent based on the flow of traffic from the 
automat ion /feeder controller interaction. Accord- 
ingly, most controllers (10) indicated that the effort 
required to set up the landing sequence was reduced. 
For the 210-knot manual procedure, subjects were 
briefed to employ the same speed control strategy 
used by the automation, that is, issue the speed re- 
duction after the turn to final. In response to this 
procedure as compared with operating practices the 
subjects were accustomed to, opinion was generally 
divided between no difference and a slight differ- 
ence. Nine subjects added comments for this ques- 
tion: three indicated that they would have slowed 
aircraft at a point prior to the turn to final, whereas 
four others indicated that they rarely use speed con- 
trol. Most subjects (11) did feel that based on the 
traffic situation during the test run, a pattern speed 
of 210 knots through turn to final was acceptable. 

5. 5. 1.3. Graphic Marker Format Questionnaire 

5. 5. 1.3.1. 1 70-knot procedure 

All the subjects agreed that it was “easy” adapt- 
ing to the graphic marker. In response to whether 


the graphic marker should continue to be displayed 
after the turn has been issued (as implemented in 
this study), five subjects said that this resulted in 
no perceivable clutter, two subjects said that this 
produced excessive clutter, and the remainder fell 
between these endpoints. The response to whether 
focus on aircraft- to- graphic turn-marker position re- 
lationship affected the aircraft-to-aircraft attention 
was that it was only somewhat affected or not af- 
fected according to slightly over half (seven) the sub- 
jects. Ten of the 12 subjects felt that there was at 
least slightly more precision in spacing of aircraft us- 
ing the graphic marker versus a manual operation 
with no automation assistance. In response to a 
question comparing the turn positions suggested by 
the turn marker versus where subjects would have 
turned the aircraft unaided, no respondents indicated 
a “strong” difference. 

5.5. 1.3.2. 210-knot procedure 

Almost all (11) of the subjects felt that it was 
“somewhat easy” to “easy” adapting to the graphic 
marker. Subjects generally felt that use of the speed 
reduction advisories provided by the graphic resulted 
in more precise spacing than they were able to pro- 
vide in a manual operation and their workload was 
reduced. Most (10) felt that focus on the speed re- 
duction advisor had little effect on attention to either 
aircraft-to-aircraft spacing or the overall traffic pic- 
ture. In response to a question comparing the turn 
positions suggested by the turn marker versus where 
subjects would have turned the aircraft, no respon- 
dents indicated a “strong” difference. Eleven sub- 
jects felt that the speed reduction points suggested 
by the program either agreed or closely agreed with 
the point where they would have issued the reduction 
had no FASA’s been provided. 

5. 5. 1.4. DICE Countdown Format Questionnaire 
5.5. 1.4-1- 170-knot procedure 

In terms of adapting to using the DICE with the 
170-knot procedure, eight subjects indicated that it 
was either “somewhat easy” or “easy” adapting to 
using the format while the other four said that it was 
“neither easy nor difficult.” All subjects agreed an 
improvement in spacing precision resulted by using 
the DICE as compared with a manual operation 
with no automation FASA’s. The subjects were 
divided on the question of whether focus on the 
DICE countdown value affected attention to aircraft- 
to-aircraft spacing; six felt that there was little or 
no effect, five felt that attention was reduced, and 
one subject didn’t feel either way. The subjects 
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were also divided as to whether it was necessary to 
have the computer suggested headings in the data 
block. Of those respondents who felt the headings 
were necessary, the indication was they were “nice 
to have” rather than necessary. The implementation 
of the DICE countdown in the simulation was based 
on an aircraft’s performance relative to its schedule 
and therefore was not a straight clock countdown. 
Subjects were asked if they would prefer another type 
of countdown. Two indicated a desire to have an 
alternate form of countdown, four preferred the type 
of countdown implemented in the study, and six were 
neutral. 

5.5.1. 4-2. 210-knot procedure 

Ten subjects felt that it was either “somewhat 
easy” or “easy” adapting to the use of the DICE 
with the 210-knot procedure. In terms of workload, 
9 subjects indicated a reduction with the DICE turn 
indication, and 10 indicated a reduction with the 
DICE speed advisory. Opinion was divided as to 
whether the DICE turn advisory affected the sub- 
jects attention to aircraft-to-aircraft spacing -nine 
felt that their spacing of aircraft wa s more precise 
when using the speed advisory versus a manual oper- 
ation with no automation FASA’s and three subjects 
did not feel that their spacing was either more or less 
precise. 

5. 5. 1.5. Centerline Slot Marker Format 

Questionnaire 

5.5. 1.5.1. 170-knot procedure 

Adapting to using the centerline slot marker at 
170 knots was evaluated by nine subjects to be ei- 
ther “somewhat difficult” or “difficult.” This evalu- 
ation is supported by a number of verbal comments 
during those data runs and during the postrun de- 
briefings. Opinions were widely divided regarding 
workload when using this format with seven subjects 


noting a workload increase. Nine subjects consid- 
ered the information provided by the slot markers to 
be “somewhat useful” or “useful.” A slim majority 
(7) of the subjects would like to have a FASA, such 
as turn advisories, vector advisories, to help them 
deliver aircraft into their slots. 

5. 5. 1.5. 2. 21 0-knot procedure 

Subjects responses were divided on the question of 
adapting to the use of the slot markers at 210 knots: 
five, “somewhat difficult”; four, “somewhat easy”; 
three “neither difficult or easy.” Most subjects (8) 
indicated that their focus on aircraft-to-slot marker 
relationship reduced attention to aircraft-to-aircraft 
spacing. Ten subjects felt that the slot markers pro- 
vided useful information. One subject commented 
that the slot markers “gave guidance as to correct 
spacing — especially good with heavy aircraft.” Seven 
of the subjects would like to have a FASA, such 
as turn advisories, vector advisories, to help them 
deliver aircraft into their slots. 

5.5.2. TLX Workload Assessment 

The principal means of assessing subject con- 
troller workload was the Task Load Index (TLX) 
procedure. The TLX is a multidimensional rating 
developed at NASA Ames Research Center (ref. 21), 
and it uses subjective ratings of six workload con- 
tributing factors that are relatively weighted by each 
subject. It is a simple, quick, and systematic process 
for compiling workload ratings. 

Example TLX questionnaires, and the individual 
controller subjects 1 format ratings and source-factor 
weightings plots are presented in section F.3. Each 
subject controller’s TLX-assesscd workload is given 
in figure 36(a) for the display format and speed pro- 
cedure tested, and the mean and standard devia- 
tion are also given for all subject workloads for each 
of the tested display formats of both pattern-speed 
procedures. They are plotted in figure 36(b). 


5.5.2. 1. TLX- Assessed Workload for 170-Knot Pattern-Speed Procedure 

A step- shaped pattern characterized the TLX workload of the formats for the 170-knot pattern-speed 
procedure in figure 36(b). The centerline slot marker had the highest workload value (63.6); the manual 
format was next (52.9); the graphic marker format appears to be the lowest (38.7); the DICE format (46.7) 
fell between that of the manual and the graphic marker workloads. A single- factor (display format), repeated- 
measure analysis of variance yielded a P value of 0.0002 for treatment differences. The results of a single factor 
(display format) repeated measure analysis of variance with Fisher PLSD post hoc test to determine whether 
differences in the TLX- assessed workload values were statistically significant for the 170-knot pattern-speed 
procedure are as follows: 
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Manual/FASA statistical comparisons: 

//Man < MCSM at 0 045 level of significance (95.5 percent confidence) 

A^MAN not> A^DICE cannot reject null hypothesis (a > 0.05) 

A%IAN > MGM at 0.01 level of significance (99.0 percent confidence) 

FASA statistical comparisons: 

A^CSM > A^DICE at 0.005 level of significance (99.5 percent confidence) 

fi csm > A i GM at 0.005 level of significance (99.5 percent confidence) 

A i DICE n °f > t l Q M cannot reject null hypothesis (a > 0.05) 

The analysis indicated there was no statistically significant difference between the TLX-assessed workload 
of the graphic marker and the DICE format or between that of the manual and DICE format with the 170-knot 
pattern-speed procedure. However, with acceptable statistical confidence (95.5 percent), we can say that the 
centerline slot marker workload was higher than that of the manual format. With high statistical confidence 
(99.5 percent), we can say that the TLX-assessed workload of the centerline slot marker was higher than that 
of either the graphic marker or the DICE countdown format. In addition, with high confidence (99 percent), 
we can say that the graphic marker workload was lower than that of the manual format. 


5. 5. 2. 2. TLX- Assessed Workload for 210-Knot 
Pattern-Speed Procedure 

From figure 36(b), for the 210-knot pattern-speed 
procedure, we see that the centerline slot marker 
(59.2 TLX workload), the manual (50.2 TLX work- 
load), and the DICE formats (44.4 TLX workload) 
have the same descending staircase relation to each 
other as was the case in the 170-knot procedure al- 
though the incremental workload differences arc less. 
Unlike the 170-knot procedure, however, the work- 
load rating of the graphic marker format is higher 
than that of the DICE format with practically the 
same value as the manual format. The graphic for- 
mat workload characteristic of the 210-knot proce- 
dure is discussed further in the next section where 
the 170- and 210-knot procedure workloads are com- 
pared. A single-factor (display format) repeated- 
measure ANOVA for the 210-knot pattern-speed 
procedure indicated the null hypothesis (all format 
workloads are equal) could not be rejected at the 
0.05 level of significance. 

5. 5. 2. 3. Comparison of TLX- Assessed Workload 
for 170- and 210-Knot Pattern-Speed 
Procedures 

Workload difference between the 170- and 210- 
knot pattern-speed procedures were tested via a two- 
factor (speed procedure, display format) repeated 
measure analysis of variance. The result was that 
the null hypothesis (workloads between the two speed 
procedures are equal) could not be rejected at the 


0.05 level of significance. Therefore the 210-knot 
pattern-speed procedure did not significantly change 
the workload of the formats as a group relative to the 
formats of the 170-knot procedure as a group. This 
finding is important. Recall from section 5.1 that the 
runway separation precision improved across all for- 
mats for the 210-knot procedure and from section 5.2 
that the vectors/aircraft were reduced for both the 
manual and centerline slot marker formats for the 
210-knot procedure. Therefore, for the experimental 
conditions, these benefits were obtained without any 
significant increase in controller workload. 

When comparing the 170- and 210-knot proce- 
dure formats, the 210-knot graphic marker workload 
was puzzling. The difference in the TLX-assessed 
workload values between the 170- and 210-knot pro- 
cedures was less than 4.5 for the CSM, MAN, and 
DICE format comparison. Further, the workload for 
the 210-knot procedure was less for each format. On 
the other hand, the TLX workload difference was 11.6 
between the 170- and 210-knot graphic-marker com- 
parison and the trend noted above was reversed with 
the 210-knot procedure having the higher workload 
than the 170-knot procedure. 

The strong trend reversal noted was caused by 
five subjects (7, 8, 10, 11, and 12) whose TLX evalua- 
tions of the graphic marker format were considerably 
higher for 210 knots than for 170 knots (fig. 36(a)). 
Relative format workload ratings by the five sub- 
jects for the separate 170- and the 210-knot proce- 
dures (section 5. 5. 3. 2) as well as the combined speed 
relative rating (section 5. 5. 3. 3) were cross-checked 
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with their TLX workload evaluation and found to 
be consistent. Four of these five subject controllers 
rated the graphic marker to be their preferred format 
in the 170-knot evaluation; however, none of these 
five subjects rated the 210-knot graphic as their pre- 
ferred format in either the 210-knot or the combined 
170- and 210-knot evaluation. 

Comments from these five subject controllers 
indicate that the perceived higher workload, for 
the graphic marker 210-knot format relative to the 
170-knot format, was the lack of conspicuousness 
of the 210-knot procedure speed reduction marker. 
The speed marker location was such that it may not 
“stand out” sufficiently from the video map depic- 
tion of the final-approach course (fig. 5). Initially 
we believed that there was a tendency to fixate on 
the speed marker. However, subsequent review of 
the oculometer data (statistical and video tapes) dis- 
proved this idea. For these five subjects, time mea- 
surements were taken to determine the amount of 
time between when a graphic first appears and when 
the controller first looks at it. The mean time was 
1.1 sec for the turn marker compared with 1.5 sec 
for the speed marker. The difference between these 
two times does not seem to be as great as subjec- 
tive comments seem to indicate. Four controllers 
stated in the format questionnaires and final debrief- 
ing comments that they would have preferred an- 
other implementation of the graphic speed marker. 
Two controllers stated that they missed the initial 
appearance of the graphic speed marker and thought 
that the speed marker should either flash or be color 
coded when it first appears on the display. Two other 
subjects stated that they thought that the speed 
marker should flash if the command was not issued 
on time. Reactions indicate that subjects thought 
that a graphic speed advisor was desirable in concept. 
However, investigation of alternatives for identifying 
the speed reduction point could be beneficial. 

5. 5. 2. 4. Source-of- Workload Weightings and 

Ratings of TLX Factors 

The TLX- assessed workload obtained from a sub- 
ject is influenced by the subjects weighting and scaled 
ratings of six workload source factors. Definitions of 
the six factors are contained in section F.3.1 and ref- 
erence 21. The process of computing the workload is 
detailed in reference 21. 

The extent to which a subject’s scaled rating, 
of each of the six contributing factors, influences 
the workload of the specific task being evaluated 
(i.e., controlling final traffic with a specific FAS A 
format) is determined by the factor weightings for the 


general type or class of activity (i.e., controlling final 
traffic) to which the specific evaluated task belongs. 
The weights, resulting from a pairwise comparison 
of the factors, indicate the relative impact each has 
on the general activity or task. Thus, the weights 
provide insight about the general task (in this case, 
controlling final traffic). The mean of the factor 
weights of all 12 subject controllers are listed in 
figure 37(a). These values indicate how, on the 
average, our group of subjects weighted the impact 
of each of the six TLX workload factors on their 
final controller function. The higher the weight, 
the more critical is the factor to a terminal final- 
controller’s workload. The mean factor weights in 
order of importance are given in the following table: 


Factor 

Weight 

Mental demand (MD) 

4.25 

Own performance satisfaction (OP) 

3.17 

Temporal demand (TD) 

3.08 

Your effort required (ER) 

2.83 

Frustration you experienced (FE) 

1.67 

Physical demand 

0 


Note, the above weights sum to 15 as is the case 
for an individual test subject’s pairwise comparison 
process. 

The rating of a factor reflects the subject’s judg- 
ment of the relative scaled magnitude of that fac- 
tor when employing the specific display format be- 
ing evaluated. The mean of the factor ratings of 
all 12 subjects are listed in figure 37(a). These val- 
ues indicate how, on the average, our group of con- 
trollers rated the six factors relative to each tested 
display format. These individual workload contribut- 
ing factor means are plotted in figure 37(b) along 
with the mean of the individual subject workloads 
for each display format. Note that physical demand 
was not plotted because of its 0 weight, which meant 
that physical demand ratings did not influence the 
TLX-assessed workload. 

Figure 38 is a depiction of the data in figure 37(a) 
in the conventional TLX format (ref. 21). Another 
form of mean workload can be obtained by using the 
mean weights and mean ratings to compute an overall 
workload by the TLX procedure as was done for the 
individual subject workloads. These alternate means 
are listed in figure 37(a) and plotted in figure 38 
for each display format. Even though this method 
is not mathematically identical to the mean of the 
subjects’ individually calculated TLX workloads, the 
two methods should give approximately the same 
values. A comparison of the data in figures 36(a) 
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and 37(a) shows the two mean values for each format 
in close agreement (maximum difference was 1.6). 

Because of the number of display format mea- 
sures in our study, we restricted our statistical evalu- 
ation to conventially calculated TLX-assessed work- 
load via repeated measure ANOVA on the data of 
figure 36(a). The TLX workload contributing factors 
could themselves be treated as dependent variables 
for statistical analysis if one were interested in fur- 
ther pursuit along this vein. A rough sense of such 
an analysis can be obtained from visual comparison 
of the mean factor ratings in figure 37. 

5.5.3. Relative Rating of Display Format 

Three relative rating questionnaires were designed 
to acquire from the subjects their relative ranking of 
the formats with respect to three specific criteria: 

1. Workload or effort required to use the format 

2. Ease of adapting to or learning to use the 
format 

3. Amount of help or benefit in spacing traffic on 
final 

The applicability of the manual format to criteria 2 
and 3 was determined to be questionable. There- 
fore, the “manual” responses were removed and the 
placement of the remaining three formats adjusted 
accordingly. Subjects completed three separate rat- 
ings, one after each of the two pattern-speed proce- 
dures and a final combined rating, for all the formats 
of both pattern-speed procedures as a group at the 
completion of all data runs. The questionnaire form 
is shown in section F.4. 

For each criterion, a rating scale (as shown below) 
consisting of ten positions with the appropriate end- 
point descriptors defining the ends of the scale (e.g., 
most workload and least workload) was provided. An 
example of a scale is as follows: 

i i i i i 

most workload least workload 

Subjects were instructed to first determine which for- 
mats represented the endpoints and then position the 
remaining formats in their relative positions along 
the scale. This approach is equivalent to taking what 
would be the difference between the lowest and high- 
est ratings on an absolute scale and blowing it up for 
resolution or differentiation. These results will have 
meaning only in a relative position sense. For exam- 
ple, a mean rating of 8 does not imply 4 times better 
than a low mean rating of 2. However, some measures 


such as workload are not always evaluated relative 
to absolute human capability but often have relative 
meaning only in the domain of interest, which in our 
case was final controller reaction or performance with 
the tested display formats relative to each other. 

5.5.3. 1. Relative Rating of Formats for 

170-Knot Pattern-Speed Procedure 

Figure 39 shows the distribution of the subjects 
according to their format rating for the workload or 
effort required to use the format for the 170-knot 
pattern-speed procedure. Nine subjects rated the 
graphic marker as having the least workload or effort 
required to use the format. Although not rated as 
high as the graphic, the majority (8) of the subjects 
felt that the DICE represented an overall reduction 
in workload relative to the manual or the centerline 
slot marker. Three subjects commented that the 
DICE format requires more concentration than the 
graphic marker. However, they further stated that 
the information is in a desirable location, that is, 
near the aircraft position symbol and the standard 
data block information. Opinion was divided for 
the manual format, and seven subjects felt that the 
slot markers generated more workload than the other 
formats. (Fewer subjects (3) evaluated the manual 
format as having the most workload than those (7) 
making the same evaluation for the slot markers.) 
For the 170-knot procedure, a comparison of the 
graph showing questionnaire mean workload rating 
and the TLX results (fig. 40) shows a strong ranking 
correlation between the two workload assessments. 

Figure 41 indicates the distribution of the sub- 
ject controllers according to their format rating cri- 
terion 2 (“ease of adapting to or learning to use for- 
mat”). Nine subjects rated the graphic as the easiest 
format to adapt to, whereas 10 subjects evaluated 
the slot markers as being the most difficult. No sub- 
jects evaluated the graphic as being the most difficult 
nor the slot markers as being the easiest format with 
which to adapt. Individual practice time did not al- 
ways indicate how difficult it was to adapt to a format 
because of the order of the pattern-speed procedure 
and the order of the formats themselves in the test 
sequence. However, for the 170-knot procedure, the 
composite mean practice time used by the subjects 
prior to the data runs correlates with and supports 
results of the “ease of adapting” rating (fig. 42). 

Figure 43 depicts the distribution of the sub- 
ject controllers according to their format rating for 
“amount of help or benefit in spacing traffic on fi- 
nal” criterion. In terms of this criterion, eight sub- 
jects found the graphic marker the most helpful, eight 
stated that the slot markers were the least helpful, 
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and ratings were divided about the helpfulness of the 
DICE, 

5. 5. 3. 2. Relative Rating of Formats for 

210-Knot Pattern-Speed Procedure 

Six subject controllers rated the graphic to have 
the least workload for the “workload or effort re- 
quired to use the format 1 ’ criterion for the 210-knot 
procedure (fig. 44) as compared with nine in the 
170-knot procedure (fig. 39). Four subjects evalu- 
ated the slot marker as having the most workload for 
the 210-knot procedure as compared with seven for 
the 170-knot procedure. Six subjects rated the man- 
ual as having the most workload for the 210-knot 
procedure as compared with three for the 170-knot 
procedure. This increase in the number of subjects 
may have been the result of the normal operational 
practices used by several subjects as stated in the 
verbal debriefing, that is, many controllers at ORF 
do not use speed control regularly to space traffic on 
final. Therefore, the additional effort in determining 
the location to issue a speed reduction may have been 
perceived as requiring more effort. Figure 40 depicts 
the relationship between the ratings for “workload or 
effort required to use the format” criterion and the 
TLX results. For the 210-knot procedure, the mean 
rating for the graphic format was the lowest of all 
formats, whereas the TLX results indicated a slight 
workload magnitude increase for the graphic over the 
DICE. However, recall there was no statistical signif- 
icant difference between TLX outcome of the graphic 
marker and DICE countdown formats. 

Again shown in figure 40, and as discussed earlier 
in section 5. 5. 2. 3, the TLX workload assessment for 
all the formats, except the graphic marker, was less 
for the 210-knot pattern-speed procedure than for the 
170-knot procedure. This increase for the graphic 
marker was attributable to five particular subject 
controllers (section 5. 5. 2. 3) whose TLX workload rat- 
ings were considerably higher for the 210-knot proce- 
dure than the 170- knot procedure. Figure 45 presents 
the distribution of the relative format workload rat- 
ings of these five subjects for the 170- and 210-knot 
procedures. The same pattern can be seen as was ob- 
served for the TLX ratings. A similar breakdown of 
the distribution of the combined display format rela- 
tive ratings for both speed procedures is included in 
section 5. 5. 3. 3 for these five subjects as well as for 
the other seven subjects. 

Figure 46 indicates the distribution of the sub- 
ject controllers according to their format rating for 
the “ease of adapting to or learning to use the for- 
mat” criterion. In this figure, the order of preference 


for the FASA formats relative to each of the three 
criteria is the same as for the 170-knot procedure. 
Essentially, most subjects (9) found adapting to the 
graphic format easiest, and the slot markers most 
difficult. However, two subjects rated the graphic 
as the most difficult to adapt to in the 210-knot 
procedure as compared with none in the 170-knot 
procedure; two rated the slot marker the easiest to 
adapt to in the 210-knot procedure, whereas none of 
the subjects rated the slot markers as the easiest in 
the 170-knot procedure. A comparison between the 
“ease of adapting to or learning to use the format” 
criterion rating and the practice time used is pro- 
vided in figure 42. Overall, nine subjects felt that 
the graphic was the easiest format to adapt to, even 
though slightly more mean practice time was used 
for the graphic than the DICE. However, there was 
not any statistical significant difference between the 
two practice times. 

Figure 47 depicts the distribution of the subject 
controllers according to their format rating for the 
“amount of help or benefit in spacing traffic on final” 
criterion. In terms of this criterion, again as in the 
170-knot procedure, the most controllers (8) selected 
the graphic format as the most helpful. However, 
the graphic marker had two ratings of least helpful. 
On the other hand, DICE had five most helpful 
but no least helpful ratings. This contrasts with 
DICE which received four least helpful ratings in the 
170-knot procedure. As a result, the means of the 
rating distributions, relative to the “amount of help 
or benefit in spacing traffic on final” criterion, were 
equal for the graphic and DICE formats. Among the 
three formats, clearly the centerline slot marker had 
the consensus rating (10 subjects) as least helpful, 
even though it received two high ratings. 

5. 5. 3. 3. Relative Rating of Combined Formats 

for Both Pattern-Speed Procedures 

After all runs for both pattern-speed procedures 
were completed, each subject controller performed 
combined format relative ratings for both pattern- 
speed procedures. These combined ratings (whose 
distributions according to subject count shown in 
figs. 48, 49, and 50) consisted of relatively rating, 
as a group of eight, all the formats together for both 
the 170- and 210-knot procedures, according to the 
same three criteria used in the individual pattern- 
speed format ratings. The results of the combined 
rating in terms of format preference were consistent 
with the ratings of the individual speed procedures 
for all three criteria: “workload or effort required 
to use the format” (fig. 48), “ease of adapting to or 
learning to use the format” (fig. 49), and “amount of 
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help or benefit in spacing traffic on final” (fig. 50). 
Additionally, in all but one case, both speed proce- 
dures for a particular format were preferred over the 
subsequent format choice (e.g., graphic 210 knots, 
graphic 170 knots, DICE at 210 knots, ...). The only 
exception, as such, was the mean workload rating 
tie between the centerline slot marker at 210 knots 
and the manual format at 170 knots. Another very 
significant finding in the combined ratings was that, 
in every criteria and display format case, the mean 
rating for the 210-knot format was higher than its 
corresponding mean rating for the 170-knot format. 

For the “workload or effort required to use the 
format 1 ’ criterion, the mean ratings between the two 
speed procedures for each format were all relatively 
close except the ones for the centerline slot mark- 
ers. This difference for the slot markers may be at- 
tributable to the overall difficulty that subjects had 
in working with the slot markers in the 170-knot pro- 
cedure. The graph of the rating for “workload or 
effort required to use the format” criterion and the 
TLX (fig. 51) shows a ranking correlation between 
the two independent ratings. 

To further support the TLX discussion of sec- 
tion 5. 5. 2. 3, a breakdown of the combined format 
workload rating between five particular subjects and 
the remaining seven subjects is shown in figure 52. 
These five subjects (7, 8, 10, 11, and 12) had a 
TLX mean rating (6.9) indicating higher workload 
assessments for their 210-knot graphic marker for- 
mat (210GM) than did the other seven controllers 
(8.8). For the group of five, the 210-knot graphic 
marker format ranked third among all formats as 
opposed to first for the group of seven. These sep- 
arate and independent, combined ratings paralleled 
those of the individual pattern-speed speed ratings 
(section 5. 5. 3. 2) and further supported the work- 
load results obtained from the TLX for the 210-knot 
pattern-speed procedure. 

Considering all the ratings for “ease of adapting 
to or learning to use format” criterion, the most 
universally held opinion for both speed procedures 
was that the centerline slot marker was the hardest 
format to adapt to using. There were no subjects 
that evaluated the graphic format on the least helpful 
end of the scale for the “amount of help or benefit 
in spacing traffic on final” criterion. The only other 
consistent result in the combined speed rating for 
this criterion was that most subjects (8) found the 
slot markers at 170 knots to be the least helpful. 

The results of the subject evaluations for “work- 
load or effort required to use the format,” “ease 
of adapting to or learning to use the format,” and 


“amount of help or benefit in spacing traffic on fi- 
nal” criteria are remarkably consistent throughout 
all formats as shown in figure 53. Overall there is 
a consensus ranking among the FASA’s, as to con- 
troller preference, relative to all three of these crite- 
ria. The graphic marker emerges as the consensus 
first choice and the centerline slot marker as the con- 
sensus last choice. In addition, the controllers pre- 
ferred the 210-knot procedure for each format, par- 
ticularly for the centerline slot marker format. Even 
though it was the first choice overall, a sizable mi- 
nority (5 of 12 subjects) downgraded the 210-knot 
graphic marker format relative to workload because 
they felt that the speed reduction marker should have 
been more conspicuous than was used in the test. 

5.5.4. Questionnaire Comparison of 170- 

and 210-Knot Procedures 

At the completion of all data runs, subjects were 
asked to evaluate, for each of the four formats, the 
level of difficulty associated with spacing aircraft on 
final using the 210-knot pattern-speed procedure as 
compared with the 170-knot procedure. This ques- 
tionnaire and responses are presented in section F.5. 
It should be noted that according to the subjects, 
speed control on final is not extensively used for spac- 
ing traffic at Norfolk Approach Control (where the 
controller subjects are based). However, when asked 
which of the two speed procedures used in the study 
more closely approximated their operation, 11 of the 
12 subject controllers said the 210-knot procedure 
more closely approximated the speed profiles flown 
by aircraft not issued speed restrictions. 

Overall, the 210-knot procedure was considered 
to be easier to use for spacing aircraft than the 
170-knot procedure. Although the reason for this 
consideration is not readily apparent from comments 
made on this questionnaire (appendix F), five sub- 
jects stated in debriefing sessions that the speed ad- 
justment provided for in the 2 10- knot procedure, 
which allows for fine tuning of spacing, w'as quite 
helpful. For the manual and the slot markers, the 
2 10- knot procedure was evaluated as harder to use 
for spacing traffic by two and zero subjects, respec- 
tively. For both the graphic and the DICE for- 
mats, the 210-knot procedure was rated harder by 
three subjects. Depending on the format, somewhere 
between 9 to 12 controllers felt that the 210-knot 
pattern-speed procedure was as easy or easier than 
the 170-knot procedure for that same format. This 
outcome follows the same pattern as seen in the com- 
bined format rating (section 5. 5. 3. 3). All subjects felt 
that vectoring to the slot markers was easier wdth the 
210-knot procedure. Debriefing comments received 
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from seven subjects indicated significant difficulty, 
and sometimes frustration, in trying to vector the 
aircraft into the slot marker when both the aircraft 
and slot markers moved at the same speed. 

5.5.5. Final Debriefing Questionnaire, 

Verbal Debriefing, and Other Controller 

Comments 

Following all the data runs, the subjects com- 
pleted a final debriefing questionnaire and partici- 
pated in a verbal debriefing. The final debriefing 
questionnaire targets the subjects’ opinions about 
the realism and adequacy of the simulation facilities. 
The verbal debriefing provides an opportunity for the 
subjects to provide final comments on specific sub- 
ject areas such as the formats themselves, possible 
concerns in using FASA’s. Additionally, the subjects 
were given an opportunity and encouraged to com- 
ment on any aspect of the simulation, FASA’s, au- 
tomation in general, or any subject area that they 
chose. The final debriefing questionnaires along with 
the subject responses are presented in section F.6. 
The format for the verbal debriefing is included in 
section F.7. 

The final debriefing questionnaire contains ques- 
tions related to the test environment and simula- 
tion. There was unanimous agreement that the 
test sessions were conducted in a controlled, seri- 
ous, and professional manner. Overall, the sub- 
jects felt that the simulation was reasonably realistic. 
Both the simulated radar display and the commu- 
nications were evaluated to be adequate for the re- 
quired task. However, the following comments were 
made by two subjects: “When instructions were is- 
sued to the wrong aircraft, pseudopilots did not ques- 
tion the calls” and “Pilots made very few errors 
and responded too quickly.” In response to the ef- 
fect of the “physical environment” on the subjects’ 
test performance, it was generally felt that perfor- 
mance was “neither improved nor degraded.” All 
subjects evaluated the initial briefings on the Denver 
airspace and procedures and the format briefings to 
be adequate. Feelings were also unanimous that the 
training received during the format practice runs was 
adequate. 

In the verbal debriefing, the subjects were asked 
for comments on each of the formats. The most pos- 
itive comments were received for the graphic turn 
marker; in general, subjects felt that the graphic was 
easy to use and provided information in an easy to in- 
terpret manner, requiring a minimal amount of adap- 
tion time. Typical comments about the graphic turn 


marker were “easy to use” and “required less atten- 
tion than DICE or slots.” The graphic speed advisor 
was well liked by all but one subject. Several sub- 
jects did, however, suggest that the speed marker 
needs to be more conspicuous. The following sugges- 
tions were offered: the speed marker should flash at 
the first appearance, flash at the time the command 
should be issued, and color could be used to accent 
the speed marker. The DICE time-to-turn advisor 
received less support than the graphic. However, the 
consensus was that it is not difficult to adapt to and 
the information is in a location which is close to other 
data block information and the aircraft position sym- 
bol. In the format questionnaire only two subjects 
indicated a preference for an alternative form of the 
DICE countdown used in the study (section 5. 5. 1.4). 
However, during the debriefing discussions, a total 
of four subjects stated a desire for a straight clock 
countdown (i.c., 8, 7, 6, 5, 4, ...). Comments on 
the DICE timc-to-speed-reduction advisor were ba- 
sically the same as those for the time-to-turn advi- 
sor. Finally, most subjects felt that the centerline 
slot marker was the most difficult of the FASA’s to 
adapt to and the most difficult to use. Four subjects 
felt that another aid such as the graphic or DICE 
could be used to assist in merging aircraft with slots. 

In response to the question of fixation (excessive 
focus on situation or location), clutter, and distrac- 
tions, most subjects stated that there were minor 
problems in at least one of the three areas with one or 
more formats. Occasional fixations were reported by 
five subjects in the use of the DICE, five subjects in 
the use of the centerline slots, and one subject in the 
use of the graphic marker. The only comments re- 
garding clutter were noted by three subjects, all with 
the graphic turn marker; the subjects stated that the 
clutter was minor and did not present a problem. 

Nine of the 12 subjects were enthusiastic about 
the overshoot prediction feature in the 210-knot pro- 
cedure. The feature provided advance warning about 
an aircraft on the base leg which the automation 
predicted would need to overshoot the localizer, to 
preserve separation, unless some additional action 
(speed reduction or vector) is taken prior to the 
turn-to-final. 

The subjects were asked several questions to so- 
licit their opinions on the use and potential impact 
of FASA’s. There was not a common denominator 
among the responses from the subjects relative to 
the question of “Did the system change the mental 
tasks involved in controlling traffic?” However, there 
were interesting comments: “Mental demand less- 

ened, lowered”; “Felt like a robot”; “Controller reacts 
to system as opposed to formulating.” When asked 
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“Do you feel that an automated advisory system will 
create any special problems with controller training 
or proficiency?,” comments were wide: “No, after 
awhile controller would work traffic like the com- 
puter”; “No, I'll take all the help I can get”; “Con- 
trollers will have to develop new expectations from 
their careers”; “[FASA’s] could extend the productive 
life of a controller.” The subjects were unanimous in 
their feeling that automated aids would be benefi- 
cial in reducing workload and in increasing spacing 
precision. Subjects were asked if they had any final 
thoughts, opinions, or suggestions that were not pre- 
viously covered; by and large, only minor comments 
were made. 

6. Major Results 

Working jointly with the FAA and in collabo- 
ration with Lincoln Laboratory, MIT, and NASA 
Ames Research Center, this study was conducted 
at the NASA Langley Research Center to gather 
comparative performance data among three can- 
didate final- approach spacing aid (FASA) formats. 
Several objective measures together with subjective 
questionnaire data were used to obtain an in-depth 
assessment. The data were gathered from the per- 
formance and reactions of 12 subject controllers pro- 
vided by the FAA. The performance measures were 
obtained in a dynamic real-time TRACON simula- 
tion with varied display formats and pattern-speed 
procedures. For each of two representative pattern- 
speed procedures (a 170-knot pattern- speed proce- 
dure and a 210-knot pattern-speed procedure with 
speed control aiding), data were collected by us- 
ing four final-controller, display-format conditions: 
manual/ ARTS III, graphic marker, DICE count- 
down, and centerline slot marker. The following sec- 
tions are a summary of the experimental results. 

6.1. Aircraft Delivery and Separation 
Precision 

The measure used to assess precision of aircraft 
delivery and separation was the standard deviation of 
aircraft-pair interarrival errors at the runway thresh- 
old. For the 170-knot pattern-speed procedure, the 
centerline slot marker format did not statistically 
improve controller delivery precision relative to the 
manual/ARTS III format value (18.9 sec). How- 
ever, both the graphic marker and DICE countdown 
formats improved controller delivery precision, over 
the manual format, by statistically equivalent in- 
crements (range of reductions from 4.2 to 5.0 sec). 
For the 210-knot pattern-speed procedure, all the 
FASA formats improved controller delivery preci- 
sion relative to the manual/ARTS III format value 


(15.4 sec). Controller use of the DICE countdown 
resulted in the most delivery improvement (7.2-sec 
reduction), the graphic marker resulted in slightly 
less improvement (6.0-sec reduction), and the center- 
line slot marker resulted in the least improvement 
(4.2-sec reduction). For all formats, operating 
with the 210-knot procedure resulted in statisti- 
cally improved controller delivery precision relative 
to the comparable 170-knot procedure format. The 
210-knot manual/ARTS III format improvement was 
3.5 sec, whereas the 210-knot improvements for the 
three FASA’s were between 5.3 to 6.6 sec over their 
respective 170-knot formats. 

In terms of delivery precision, for both the 
170- and the 210-knot pattern-speed procedures, the 
graphic marker and DICE countdown formats are 
both superior to the centerline slot marker format. 
The graphic marker and DICE countdown gave sim- 
ilar precision results with the DICE having a minor 
edge in the 210-knot procedure. The measured im- 
provement in delivery precision obtained from the 
final- region speed-reduction cueing of the 210-knot 
procedure, relative to the constant speed of the 
170-knot procedure, was a significant finding. This 
indicates that the application of speed control aid- 
ing, on final approach, to a FASA has the potential 
to significantly improve aircraft separation at facili- 
ties where an initial higher pattern speed on final is 
practical. A simple analysis of runway arrival rate 
indicated that the improved precision of a FASA, 
such as the graphic marker, has the potential to in- 
crease the TATCA IMC runway arrival rate over that 
of a TATCA system without a FASA. The magni- 
tude of the increase depends on which pattern-speed 
procedures are assumed in the comparison. For ex- 
ample, the arrival rate increase was 6.6 percent for 
the 170-knot procedure, 10.6 percent for the 210-knot 
procedure, and a more dramatic 16.5 percent when 
the manual format for the 170-knot procedure was 
compared with the 210-knot pattern-speed procedure 
with its turn and speed reduction aiding. 

6.2. Vectors Per Aircraft Issued in Final 
Sector 

Data were gathered on the mean number of vec- 
tors per aircraft issued by the final controller in merg- 
ing and spacing traffic for final approach. For the 
170-knot procedure, all the FASA formats reduced 
the mean number of vectors per aircraft relative 
to the manual/ARTS III format value (2.7 vectors/ 
aircraft). Both the graphic marker and DICE count- 
down formats reduced the mean vectors per air- 
craft by equivalent increments (1.1 vectors/aircraft), 
whereas the centerline slot marker had less reduction 
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(0.5 vector /aircraft). The results for the 210-knot 
procedure were similar, where all the FASA for- 
mats also reduced the mean number of vectors 
per aircraft relative to the manual/ARTS III for- 
mat value (2.2 vectors/aircraft). For the 210-knot 
procedure, both the graphic marker and DICE 
countdown also reduced the mean vectors per air- 
craft by equivalent increments (0.7 vector/aircraft), 
whereas the centerline slot marker had less reduction 
(0.5 vector/air craft). When the 210-knot procedure 
is compared with the 170-knot procedure, the 210- 
knot procedure reduced the mean number of vectors 
per aircraft by equal amounts (0.4 vector /air craft) 
for both the manual/ARTS III and centerline slot 
marker formats relative to their 170-knot procedure 
counterparts. However, the graphic marker and 
DICE countdown formats had no significant change 
between the two pattern-speed procedures. 

For the Denver approach routes modeled and the 
pool of subject controllers used, the graphic marker 
and DICE countdown format both gave equivalent 
larger reductions in mean vectors per aircraft (42 per- 
cent for 170 knots and 32 percent for 210 knots), 
relative to the corresponding manual/ARTS III for- 
mat, than the centerline slot marker (19 percent for 
170 knots and 20 percent for 210 knots). Therefore, 
a TATCA system with a final-approach spacing aid 
not only has the potential to improve delivery pre- 
cision but also could potentially reduce the average 
vectors per aircraft in the final region. The extent 
of the vectoring reduction would depend on the spe- 
cific TRACON geometry and procedures. Note that 
a reduction in the number of vectors issued would 
have the additional benefit of reducing somewhat the 
communication channel congestion. 

6.3. Controller Response Time to Direct 
FASA 

FASA’s which have both a suggested advisory 
and delivery time were classified as direct aids in 
this report. Controller response time to a direct 
aid is the difference between the indicated delivery 
time and the actual turn or speed message delivery 
time. Histograms of controller response time were 
plotted to the FASA basc-to-final turn indication 
and, in the 210-knot procedure, to the reduction- 
to-170 knot FASA indication for the graphic marker 
and the DICE countdown formats. These response 
time models have potential application in advanced 
system analysis and ATC simulation modeling where 
automated direct aiding is a feature of the system 
under study. 

The standard deviation of response time for 
the base-to-final advisory of the DICE countdown 


(3.3 sec) was slightly less than that of the graphic 
marker (3.7 sec). This small, though statistically sig- 
nificant difference, appears to account for the slight 
delivery precision advantage of the DICE countdown 
format. It should be emphasized that a balanced con- 
sideration of all factors, not only separation precision 
or response time, should dictate format selection. For 
instance, lookpoint data and controller comments in- 
dicated some tendency toward fixation on the DICE 
countdown turn advisory, which probably explains 
its observed response time edge. However, because 
of the potential adverse effect fixation could have 
on scanning behavior, controllers tended to rate the 
DICE countdown format below the graphic marker 
format. 

6.4. Lookpoint Measurement 

The Langley oculometer system was used to 
gather data on subject controller eye- scan behavior 
as a function of the tested display formats. Of the 
many proposed lookpoint measures, three were se- 
lected for analysis. The first measure is the amount 
of time the oculometer had the subject in-track which 
can be treated as follows: as a percentage of exper- 
imental test time, as a percentage of time divided 
among display object types, or as a percentage of 
time divided among regions of the controller’s dis- 
play. The second measure is average dwell time by 
display object type. The third measure is the num- 
ber of cross-check scans, which indicates the number 
of alternating, uninterrupted fixations between two 
display objects. 

For the oculometer in- track time measurement, 
the working hypothesis relative to task difficulty is 
as follows: the more difficult the task, the higher is 
the in- track time because there is less discretionary 
or spare time to look away from the display. For 
the 170-knot pattern-speed procedure, the graphic 
marker format had significantly lower in-track time 
than all the other formats. No significant differences 
in percentage of in-track time were found among the 
display formats of the 210-knot procedure. These 
results closely agreed with and supported the TLX 
workload analysis. 

As might have been expected, changes in the scan 
behavior were observed for all FASA formats when 
compared with the manual format. Less time was 
spent on the conventional aircraft position symbol 
and data blocks and this time was transferred to the 
aid presentation. The changes in scan behavior rel- 
ative to display zones confirmed what was observed 
for the display object measures. The location of the 
centerline slot markers, along the extended runway 
centerline, resulted in more total time in that region 
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than the other formats. On the other hand, more 
time was spent in the base and downwind regions 
where the turn aids were located, with the graphic 
marker and DICE countdown formats than was spent 
in those regions with either the manual or centerline 
slot marker. 

The subject controllers spent significantly less 
time scanning the area inside the final-approach fix 
with the centerline slot marker than was spent with 
the other three formats. This characteristic does not 
appear to be desirable when one considers that the 
delivery precision of the centerline slot marker was 
the worst of the tested FASA’s. The graphic marker 
and DICE were providing almost equivalent infor- 
mation but in different forms. The longer average 
dwell time of fixation for the DICE countdown aid 
compared with that for the graphic marker suggests 
a more efficient information transfer process for the 
graphic marker format. 

For both pattern-speed procedures, all three aids 
had significantly less cross-checking involving the 
base and extended-runway-centerline regions than 
the manual format . The graphic marker had the least 
cross-checks involving all the display regions evalu- 
ated and had significantly less cross-checks than all 
the other formats for the base and extended center- 
line regions. For the cross-checking measurement, 
the model assumed that the controller’s purpose for 
examining relative position (i.e., cross-checking) was 
to either perform some control action or monitor sep- 
aration. The hypothesis was that a reduction in the 
number of cross-checks primarily indicated a reduc- 
tion in the amount of comparison or judgment re- 
quired to properly time a control action if the amount 
of monitoring is assumed to be relatively constant. 
Accordingly, the results indicate a graphic marker 
advantage, relative to required comparisons, in mak- 
ing control action judgments. 

6.5. TLX Workload Assessment 

The Task Load Index (TLX) procedure was used 
to collect relative workload data among the tested 
formats. For the 170-knot procedure, the centerline 
slot marker significantly increased rated workload 
relative to the manual/ARTS III format, whereas the 
graphic marker format significantly reduced the rated 
workload relative to the manual/ARTS III format. 
The rated workload of the DICE countdown format 
fell between that of the manual/ARTS III and the 
graphic marker with the difference, relative to either, 
not statistically significant. 

For the 210-knot pattern-speed procedure, the 
workload rating differences among the formats were 


not statistically significant. For the centerline slot 
marker format, this reflects that the additional con- 
trol (obtained via the 210-to-170-knot speed reduc- 
tion) coupled with the inherent speed reduction cue- 
ing (given by the slot marker positions) appear to 
relieve the 170-knot speed procedure pressure of hav- 
ing to make such precise turns to align the aircraft 
in their slot marker. The requirement of an ad- 
ditional speed reduction could have been perceived 
as an added workload for the other formats of the 
210-knot procedure. However, a comparison of the 
170- and 210-knot procedures indicated there was no 
statistically significant difference in the overall rated 
workload between the two pattern-speed procedures. 
Note that a sizable minority (5 of 12) of the subjects 
perceived their workload with the 210-knot graphic 
marker format to be higher than their workload with 
the 170-knot graphic marker format because they felt 
the speed reduction marker should have been more 
conspicuous than implemented in the test. 

6.6. Questionnaires and Debriefing 
Findings 

Overall comments indicated enthusiasm for the 
use of FASA’s to improve the final controller’s per- 
formance. Most of the subjects (between 9 to 12 for 
both speed procedures) felt that the advisories pro- 
vided by the graphic marker and the DICE count- 
down display formats resulted in more precise spac- 
ing than they were able to accomplish unaided. 
Responses indicated some reduction in attention to 
aircraft-to- aircraft spacing for the graphic and DICE 
formats relative to the manual procedure. The re- 
sponses between “affected” and “strongly reduced” 
were: 5 for 170GM, 5 for 210GM, 6 for 170DICE, 
and 7 for 210DICE. In terms of clutter, the con- 
sensus was that the additional information provided 
on the display by the FASA’s did not present a 
problem; Ihis was true even for the centerline slot 
markers which were continually displayed along the 
final-approach course up to the final- approach fix. 
Relative to preference for the DICE or for a straight 
clock countdown, two responses preferred the clock 
countdown, four preferred the DICE, and the remain- 
ing six indicated either countdown would be accept- 
able. For both speed procedures, adapting to using 
the formats was easy for all subjects in the graphic 
marker case and most subjects (8 at 170 knots, 
10 at 210 knots) for the DICE. Adapting to us- 
ing the slot marker was difficult for half the sub- 
jects in the 210-knot procedure and for most sub- 
jects (nine) in the 170-knot procedure. Most subjects 
(9 at 170 knots, 10 at 210 knots) felt the slot marker 
provided useful information. However, the task of 
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merging aircraft with their slots was considered to be 
difficult, particularly in the 170-knot procedure. Ac- 
cordingly, a slim majority (seven) felt that advisories 
to assist in merging aircraft with their slots would 
be helpful. For both the 170- and 210-knot proce- 
dures, eight subjects indicated that their attention 
to aircraft-to-aircraft spacing was between “affected” 
and “strongly reduced” when using the slot marker 
display format. In the manual/ARTS III format, for 
both speed procedures, all subjects felt that the land- 
ing sequence was apparent based on the flow from the 
automation/feeder. Recall that the traffic flow was 
organized and regulated by the automation prior to 
the final sector. Accordingly, most controllers (10) 
felt that little effort was required to “set up” the 
landing sequence. 

Questionnaires for relatively rating the display 
formats (formats of only the 170-knot procedure, for- 
mats of only the 210-knot procedure, and all the for- 
mats of both the 170-knot and 210-knot procedures 
together) were designed to acquire from the subjects 
their relative ranking of the formats with respect to 
three specific criteria: “workload or effort required 
to use the format,” “ease of adapting to or learning 
to use the format,” and “amount of help or benefit 
in spacing traffic on final.” In all cases, the follow- 
ing same FAS A order of preference resulted: graphic 
marker, DICE, and slot marker. Additionally, when 
rating all the formats of both the 170- and 210-knot 
procedures together, in every format case, the mean 
ratings indicate that the 210-knot format was pre- 
ferred over the corresponding 170-knot format. This 
result was consistent with that of the comparison of 
difficulty question for the two speed procedures of 
each display format. The number of controllers rat- 
ing the 210-knot procedure as equal to or less dif- 
ficult than the 170-knot procedures were 12 for the 
slot marker, 10 for the manual, 9 for the graphic, and 
9 for the DICE. 

Final questionnaire responses and debriefing com- 
ments indicated that, overall, subjects felt that the 
simulation was reasonably realistic and that both the 
airspace/procedural and format briefings were ade- 
quate. Subjects strongly reiterated their support and 
enthusiasm for having a computer aid them in per- 
forming the final controller’s job. In terms of the 
formats, the most positive comments were received 
for the graphic turn marker. In general, subjects felt 
that the graphic marker was easy to use and provided 
information in an easy to interpret manner, requiring 
a minimum amount of adaptation. The slot marker 
was generally perceived to be the most difficult of 
the aids to adapt to, the most difficult to use, and 
required the most concentration. In terms of the po- 


tential effects of using FASA’s, concerns were raised 
that controllers would react to the FASA’s instead 
of formulating their own plan and thereby become 
“robots.” Other comments reflected a wide range of 
ideas on the potential long-range impact of automa- 
tion aids. Several subjects felt that their professional 
careers would be prolonged because of workload and 
stress reduction. However, several others felt that 
pride and job satisfaction would be taken away, and 
the possibility was raised that the use of automation 
aids might require changing the personality type of 
individuals that are recruited to be controllers. 

7, Concluding Remarks 

Measured results resolved the TATCA issue of 
whether a FAS A is beneficial if the TATCA automa- 
tion (CTAS) has already organized and tentatively 
spaced arrival traffic prior to the final sector. Some 
FASA’s significantly improved the runway delivery 
precision and reduced the average vectors per air- 
craft relative to the unaided format when both the 
FASA and the unaided format had TATCA automa- 
tion act on the traffic before the final sector. Depend- 
ing on which pattern-speed procedures are assumed 
in a basic single runway arrival benefit analysis, the 
improved precision of a FASA, such as the graphic 
marker, has the potential to increase the TATCA 
IMC arrival rate somewhere between 6 and 16 per- 
cent over that of a TATCA system without a FASA. 
Additionally, for the slower pattern-speed procedure 
tested, there was significant workload reductions for 
the graphic marker format relative to the unaided 
case. Among the formats tested, the above poten- 
tial benefits appear feasible without the subject con- 
trollers perceiving a problem with the possible draw- 
backs of FASA interface, namely display clutter and 
fixation. 

If FASA’s could improve the final controller’s per- 
formance, then which offers the most benefit? Two 
types of direct aid (graphic marker and DICE count- 
down) and one type of indirect aid (centerline slot 
marker) were tested. The graphic marker and DICE 
countdown aids were superior in measured perfor- 
mance and also preferred by the controller subjects 
over the centerline slot marker aid. Although the 
objective measures of performance for the graphic 
marker and for the DICE countdown formats were 
close in value, however, the subjective evaluation in- 
dicated a consensus preference for the graphic marker 
format. 

Experimental findings are relevant to the issue 
of what pattern-speed procedure is used in the final 
approach area of a TATCA-aided TRACON. Two 
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representative pattern-speed procedures (170 knots 
and 210 knots with speed control aiding on final) 
were tested. The slower speed procedure was rep- 
resentative of many high density TRACON’s where 
the practice, in heavy demand periods, is to reduce 
aircraft to slower pattern speeds before the turn- 
to-final or even earlier in the approach in order to 
achieve aircraft performance compatibility and pro- 
vide planning time to organize traffic. At less heavily 
loaded terminals or at high density terminals during 
nonpeak periods, aircraft are often kept at a higher 
pattern speed, until after turned to final, before be- 
ing slowed to a lower pattern speed. The experi- 
mental approach taken was to simulate this higher 
pattern-speed profile with dense traffic and, in the 
FASA formats, use automation to indicate the de- 
sired position or time, on final, to perform the nom- 
inal 2 10- to- 170-knot speed reduction. There were 
significantly improved delivery precisions measured 
with the 210-knot pattern-speed procedure for every 
format relative to its corresponding 170-knot format 
precisions. Even though requiring an extra speed- 
reduction control action per aircraft approach, as a 
group the 210-knot procedure formats did not have 
heavier TLX-determined workload than the 170-knot 
procedure formats. In addition, via three separate 
ranking criteria (workload, ease of adapting, and 
benefit to spacing), when all the formats of both 
speed procedures were ranked together as a group, 
the mean subject ratings of the 210-knot formats 
were preferred over their corresponding 170-knot for- 
mats. Note that further study of the graphic marker 
speed reduction symbol is needed, since a significant 
minority (5 of 7 controllers) felt that the workload for 
the 210-knot graphic format was heavier than for the 
170-knot graphic because the speed symbol was not 


sufficiently conspicuous. Overall, these findings indi- 
cate the potential for future consideration of FASA 
speed control aiding at facilities where higher pattern 
speeds are practical after the base-to-final turn. 

This study employed several objective and sub- 
jective measures to gain an in-depth and broad per- 
spective on relative FASA performance and also to 
cross-check findings. In this pursuit, the applica- 
tion of an oculometer to gather controller lookpoint 
data was somewhat of an innovation in reported 
ATC display research. Unlike fixed-position cock- 
pit instruments, in previous NASA pilot scan stud- 
ies, aircraft locations are constantly moving across 
the controller’s display. Postprocessing algorithms 
were developed to correlate stored lookpoint coordi- 
nates with corresponding displayed information on 
the PPI from ATC simulation data. ATC specific 
lookpoint measures were also defined. These tech- 
niques and software were detailed in a separate re- 
port (NASA CR- 191559) for possible use in other 
ATC display research. Worth mentioning was the 
reaction of the subject controllers to the oculometer 
technology upon viewing video tapes from an earlier 
data run of theirs with their own lookpoint electron- 
ically superimposed on the PPI scene. They were 
fascinated with observing their eye-scanning behav- 
ior and could instantly describe what was happening 
and why. All subjects suggested this capability had 
great potential as a diagnostic and teaching tool in 
the training of air traffic controllers. 


NASA Langley Research Center 
Hampton, VA 23681-0001 
December 20, 1993 


49 



Appendix A 

Implementation of 210-Knot Pattern-Speed Procedure 
A.l. General Description 

The TIMER (traffic intelligence for the management of efficient runway-scheduling) model is discussed 
in reference 6 for constant- approach pattern-speed operation. This appendix addresses the enhancements to 
the TIMER simulation program employed to provide realistic ffight-tiine estimates for higher pattern speed 
operation with a timed speed reduction following the base-to-final turn. Without these algorithm enhancements, 
successive direct course error (DICE) value computations on which the final-approach spacing aids (FASA) 
displays studied are based are unstable in certain situations. 

A.2. Base-Leg Vector-DICE Algorithm in 210-Knot Procedure 

A DICE advisory algorithm in its basic form provides a numeric feedback to help an air traffic controller 
to time the delivery of a control message (e.g., a turn vector message) to the pilot of an aircraft under his 
or her control for meeting a schedule. Thus, a vcctor-DICE algorithm computes, on a regular time interval, 
the estimated time of arrival (ETA) at the runway threshold assuming the vector control message is to be 
issued immediately. This ETA is, in general, earlier than the scheduled landing time (SLT) as vcctor-DICE 
calculations usually start well before the time to turn. Their difference, SLT — ETA, gives the error in arrival 
time at the runway threshold called the DICE value (DICEV) which would result if the vector turn was made 
immediately. Each successive ETA computation gives a smaller DICEV, and this trend of decreasing DICEV 
prepares the controller for the delivery of the control message to coincide with the point when DICEV reaches 
zero. 

To add a graphical aid to the numerical DICE advisory mechanism, a graphical marker needs to be placed 
at a scheduled point to turn, that is, at the point where DICEV reaches zero. A simple way to estimate 
the location of this point with respect to the aircraft position is to use the concept of ETA _G AIN, which is 
defined as the amount of ETA increase for each unit of elapsed time while the turn is delayed. In other words, 
ETA.GAIN gives the slope of the changing DICEV with respect to elapsed time. Therefore, at the current time, 
TJSTOW, given the ETA -GAIN, the current ground speed of the aircraft VH, and the current computation of 
its ETA and SLT, the amount of time that must be delayed in issuing the vector command can be estimated 
by 

T-TO-GO = (SLT - ETA) /ETA.GAIN = DICEV /ETA.GAIN (Al) 

And, the position of the graphical marker to issue the vector command is at a distance, D_TO_GO, ahead of 
its current position on the projected path: 

D-TO.GO = VH(T.TO-GO) = VH (DICEV) /ETA .GAIN (A2) 

ETA.GAIN can be estimated in either of the following two ways: 

1. ETA.GAIN can be estimated from the change of the path geometry as the result of the turn. For example, 
in the downwind DICE where the downwind leg is parallel to the final- approach path but is flown in the 
opposite direction, the gain is 2.0 if the speed is the same on the part of both legs which are lengthened 
because of the delay of the turn. If the speed on the final leg is less than the speed on downwind leg, as 
is true with the 210-knot approach pattern speeds when the aircraft is abeam of the minimum intercept 
point (MIP), the gain is increased by an amount due to speed difference, that is, 2 + (VH — VL)/VL, where 
VH is the aircraft ground speed on the downwind leg and VL is the ground speed on the final approach. 
However, an enhanced speed profile in the final approach region in the 210-knot version (to be described 
later) makes significant deviations from this calculation especially for ETA.GAIN in the base-leg- vector 
DICE computations. 

2. ETA.GAIN can be estimated from the differences of ETA’s computed at (any) two different times under 
the same condition 

ETA.GAIN = (ETA.l - ETA.2)/(TIME.l - TIME.2) (A3) 
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The second method is better than the first approach as ETA computations incorporate all the specific details 
on path geometry and speed profile of an aircraft. However, taking differences makes ETA_GAIN sensitive to 
any system noises such as those in the radar-acquired positions and ground speeds of the aircraft used as initial 
conditions in ETA computations. For example, consider the case in which ETA’s computed on successive scans 
are used in estimating the ETA _G AIN in the DICE turn on the base leg where the geometric path gains are 
near 1.0 and flight time to the runway is on the order of 200 sec. A mere 1-percent error in the estimated flight 
time due to radar position noise from one scan to the next will result in a 2-sec noise in ETA differences for 
each 4-sec scan and that gives a 50-percent noise level in the computed ETA_GAIN. Therefore, when ETA’s 
from successive scans are used for estimating ETA .GAIN’S in TIMER, a smoothing algorithm is applied to 
filter out influences from the radar noise. 

This basic approach of using ETA .GAIN to estimate the position of the graphical marker is usable as long 
as the assumption of a linear model of ETA change with respect to time is valid. For base-leg vector-DICE in 
the 210-knot version, the validity of this assumption is seriously eroded by the joint influence of two factors: 
(1) predicted overshoot of the ILS localizer and (2) speed reduction after the base-to-final turn. 

The first factor happens in those cases where the aircraft under the base-leg DICE advisory is so early 
with respect to its schedule that an overshoot across the final-approach path is predicted to delay the aircraft’s 
arrival at the runway threshold. While the base-leg DICE path geometry normally calls for a 20° ILS intercept, 
overshoot paths are variable-angle ILS intercepts aiming at the gate in 5° increments up to a 35° intercept 
depending on the amount of delay needed. Thus, the values of ETA .GAIN in the overshoot regions are 
significantly different from those in the normal (undershoot) DICE region; furthermore, it is not even linear. 
Figure A1 is a generic representation of the relationship between ILS-intercept paths and the ETA’s that might 
be computed using those segments. The left side of the figure shows several possible 20° intercept segments 
that might be used in successive ETA updates. The change in ETA’s for these segments is linear with respect 
to time as can be seen in the right side of the figure. The darker shaded lower part of the figure shows the 
relationship between ILS overshoot segments and the nonlinearity of resulting ETA’s. Referring to figure Al, 
if the normal ETA .GAIN’S (computed from successive ETA updates using the 20° intercept paths while the 
aircraft is still flying toward final) are used to predict the time to turn (intersection of the dashed lines), the 
turn marker would be placed in an overshoot position farther ahead of the aircraft than necessary. As the 
aircraft flies across the localizer centerline and ETA .GAIN’s start to pick up values closer to the true gain 
(nonlinear ETA gain region of fig. Al), the position of the turn marker shifts toward the aircraft to the correct 
location. Sometimes, large and random noises in initial ETA .GAIN’s cause the marker position to shift wildly 
in a very annoying manner. 

The second factor is due to the enhanced speed-reduction profile in the final approach legs (including the 
ILS intercept leg) designed to enhance the controllability of the speed-DICE advisory which follows the base-to- 
final vector-DICE advisory in the 210-knot procedure. The mandatory speed-reduction point is set at the MIP, 
2.0 n.mi. upstream from the gate. In order to allow aircraft to maintain higher speeds on the final-approach 
leg for as long as reasonable while reserving some catch-up capability, vector-DICE ETA computations (all 
downwind DICE and all base-leg DICE except when overshooting final is predicted) assume a nominal speed- 
reduction (to 170 knots) point at 2.0 n.mi. from the MIP (i.e., 4.0 n.mi. from the gate). This reserves about 
8 sec of catch-up capability — the amount of ETA difference if an aircraft maintains 210 knots all the way to 
MIP instead of decelerating at the nominal speed- reduction point. This also, in all except for the minimum 
eastern arrival base leg, allows 3 n.mi. or more from the end of base-leg DICE turn to the nominal speed- 
reduction point for delay capability. This allows the speed-DICE marker to be positioned well ahead of the 
aircraft after the base-to-final turn for the controller to respond to. However, if an aircraft on its base leg is so 
early that TIMER predicts and advises that overshooting the final is needed for delay, it makes sense that some 
or all of the delay capability in the speed-DICE phase should be used on the base leg to reduce the amount of 
overshoot. Thus, if all delay capability in the speed-DICE phase is to be sacrificed for the sake of reducing the 
amount of overshoot, the ETA computation would assume that the aircraft is to reduce speed right after the 
DICE turn instead of at the nominal speed reduction point farther down the path. This usage of a different 
speed-reduction profile in the overshoot region from that in the undershoot region causes the ETA .GAIN in 
the overshoot region to deviate farther from that in the undershoot region. 

To achieve the goal of deriving a stable and accurate DICE advisory, the following multistep refinements in 
the base-leg vector-DICE algorithm are implemented in the 210-knot pattern-speed procedure: 


51 



Step 1: At the start of the base- leg DICE advisory and all subsequent advisories while an aircraft is still in 

the normal undershoot region (i.e., while the aircraft is still flying toward final), compute the ETA 
with the assumption that the aircraft is to make the base-to-final turn immediately — this is the same 
ETA computed in the basic vector-DICE. Use the nominal speed-reduction point in the speed profile 
for the final-approach leg. If this ETA is close enough to the SLT of the aircraft, say within one-half of 
a scan interval, then the recommended vector turn is imminent and no more computation is needed. 
If this ETA is later than the SLT, then the aircraft has already passed its recommended vector-turn 
position and no more computation is needed. Otherwise, proceed to compute additional ETA’s. If 
the aircraft has already flown across the final-approach course and is in the overshoot region; proceed 
to step 4. 

Step 2: Compute ETA.FINAL.VL with the assumption that the aircraft continues its present course until 
reaching a point in time T_FINAL when a direct turn onto the final- approach course can be made. If 
its present speed VH is above VL (170 knots), then assume an immediate speed reduction to 170 knots 
right after the turn. ETA.FINAL.VL gives the latest time the aircraft can make without overshooting 
the final. 

Step 2.1: If the SLT is later than ETA.FINAL.VL, then overshooting the final approach course is 

predicted; proceed to step 4. 

Step 2.2: Otherwise, compute ETA.FINAL.VH at T.FINAL, with the assumption that a speed 
reduction at the nominal speed- reduct ion point. (If VH is less than or equal to VL, then 
ETA.FINAL VH is the same as ETA.FINAL.VL.) 

Step 2.2.1: If the SLT is between ETA .FINAL _VH and ETA.FINAL.VL, then T .FINAL 

is where the turn marker advisory should be placed and 

T.TO.GO = T.FINAL - T.NOW (A4) 

Proceed to the final step 5. 

Step 2.2.2: Otherwise, proceed to step 3. 

Step 3: The SLT is between the ETA (turn to 20° ILS intercept at T .NOW) and ETA.FINAL.VH (turn to 
final at T.FINAL). Compute the ETA-gain in the undershoot region by interpolation: 

ETA.GAIN.UNDERSHT = (ETA .FINAL _VH - ETA)/(T_FINAL - T.NOW) (A5) 

From the ETA-gain, compute the time-to-go before the turn should be initiated by 

T.TO.GO = (SLT - ETA) /ETA.GAIN.UNDERSHT ( A6) 

To make certain that this is a usable solution, compute the ETA, ETA.SOLN, assuming that 
the aircraft is to turn at T.SOLN = T.NOW + T.TO.GO and assuming the nominal speed pro- 
file on final. If ETA.SOLN is within the threshold delta of SLT, then proceed to step 5. 
Otherwise, the ETA.GAIN.UNDERSHT and T.TO.GO is refined one more time by using either 
[(ETA, T.NOW), (ETA.SOLN, T.SOLN)] pair or [(ETA.SOLN, T.SOLN), (ETA.FINAL.VH, 
T.FINAL)] pair, depending on whether the SLT falls on the interval (ETA, ETA.SOLN) or on 
(ETA.SOLN, ETA.FINAL.VH); that is, 

Step 3.1: If ETA < SLT < ETA.SOLN, then compute 

ETA.GAIN.UNDERSHT = (ETA.SOLN - ETA)/(T.SOLN - T.NOW) (A7) 

T.TO.GO = (SLT - ETA) /ETA.GAIN.UNDERSHT (A8) 
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Step 3.2: Otherwise, ETA_SOLN < SLT < ETA.FINAL.VH; compute 


ETA _G AIN.UNDERSHT = (ETA .FIN AL _VH - ETA _S OLN) / (T_FIN AL - T_SOLN) (A9) 


T.TO.GO = (SLT - ETA.SOLN) /ETA_G AIN_UNDERSHT (A10) 

In either ease, proceed to step 5. Note that with adequate computing power, this iterative 
process can be continued to improve the solution. 


Step 4: This step is taken when SLT is found to be later than ETA_FINAL_VL in step 2 or when the aircraft 
is already in the overshoot region and its SLT is still later than its ETA. Compute ETA _MAX at 
TJVIAX assuming the maximum overshoot path (a 35° ILS intercept at the gate) and slowest speed 
profile (deceleration right after the base- to- final turn). 


Step 4.1: 


If the SLT is greater than ETA .MAX, then ETA-MAX is the best solution. In this case, 
set 

T.TO.GO - T_MAX - T.NOW (All) 


Step 4.2: Otherwise, ETA < SLT < ETA_MAX. Interpolate to find the overshoot path (closest 
5° interval) to meet its SLT : 


ETA-GAIN _OVERSHT = (ETA.MAX - ETA)/(T_MAX - T.NOW) (A12) 


T.TO.GO = (SLT - ETA) /ETA _G AIN.OVERSHT (A13) 


Step 5: Having computed T.TO.GO, delta time from T.NOW, the final step is to compute the distance ahead 
of the aircraft D.TO.GO where the turn marker is to be placed and the equivalent DICE value at 
T.NOW: 

D.TO.GO - VH(T.TO.GO) (A 14) 

DICEV - T.TO.GO(ETA.GAIN) (A15) 

where ETA .GAIN is either ETA .GAIN .UNDER SHT or ETA .G AIN.OVERSHT, depending on which 
region the SLT falls. Although the DICEV is no longer equal to SLT - ETA, it gives a steadier 
countdown and reaches zero at the solution point. 

This refinement yields a more stable and correct turn marker position because 

1. A more correct ETA .GAIN is used in deriving the solution 

2. ETA .GAIN is computed by using ETA’s computed in the same scan; the radar noise in parameters such 
as aircraft position and ground speed has the same influence in the ETA computations, and when a 
difference of ETA’s is taken, the effect of the noise tends to cancel out yielding the change of ETA due 
to path differences only. Also, the ETA differences are taken over a larger time interval than the scan 
interval which further stabilizes the results. 


A. 3. Speed-Reduction Advisory Algorithm in 210-Knot Procedure 


The speed-DICE advisory mode is invoked by TIMER control logic on an aircraft that 

1. Has completed its base-to-final turn to either an ILS intercept heading or the final-approach heading 

2. Has not crossed the minimum intercept point (MIP) on final 

3. Has an indicated air speed greater than 170 knots 
The speed advisory consists of either 

1. A speed-DICE count-down with the value of zero marking the time when the controller should issue a 
speed-reduction message to the pilot 

2. A circular graphical marker on the projected path of the aircraft under control marking the point where 
the speed-reduction message should be delivered 
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The spced-DICE countdown value, called DICESP, is recomputed during each equivalent radar sweep update 

1. The ETA at the runway assuming an immediate (after a predefined verbal message delivery time) 
deceleration to 170 knots 

2. The difference between the ETA and the TIMER-maintained SLT which gives the current DICE value; 
that is, 

DICESP = - (SLT -ETA) (A16) 

Note that the difference (SLT - ETA) for speed- reduction advisories counts up instead of down because as 
the speed reduction is delayed, successive ETA’s decrease instead of increase as in vector-DICE updates. 
Therefore, DICESP’s are negated to maintain the “countdown” philosophy consistent with vector-DICE 
updates. 

The position of the speed graphical marker is derived on each scan by the usage of the concept of ETA-gain 
due to speed change, the computation of which is described as follows: 

1. Let VH be the current ground speed and VL be the ground speed of 170 knots IAS at a projected 
altitude of 7200 ft at the gate (note that VH must be greater than VL for the speed-DICE mode to be 
activated) 

2. Let T_DECEL and D-DECEL be the time and distance needed for the deceleration from VH to VL 

3. Let D -DELTA be the distance from the current position of the aircraft to be traveled at VH before the 
deceleration from VH and VL is to take place 

4. Let T.TO.RNWY be the remaining estimated flight time (to runway) after D_DELTA and D.DECEL 

5. Let T NOW be the current time (adjusted by the message delivery time) 

Then, the ETA for a delayed deceleration ETA VI 1 is 

ETA.VH = T_NOW + (D.DELTA/VH) + T.DECEL + T-TO-RNWY (A 17) 

On the other hand, the ETA based on immediate deceleration from VH to VL is 

ETA = T.NOW + T.DECEL + (D -DELTA / VL) + T.TOJINWY (A18) 

Therefore, the change in ETA brought about by delaying the issuing of the deceleration command (until after 
I) DELTA) is 


ETA .DELTA = ETA - ETA VH 

= (D -DELTA /VL) - (D.DELTA/VH) 

= D -DELTA {(VH - VL)/[VL(VH)]} (A19) 

The time delay corresponding to D -DELTA, T_DELTA, is simply the amount of time for the aircraft to 
travel I) DELTA at the present speed VH, that is, 

T-DELTA = D DELTA/VH (A20) 

Thus, ETA-GAIN, defined as the amount of ETA-change due to delaying the deceleration command by a unit 
time, is 

ETA-GAIN = ETA-DELTA/T-DELTA = (VH - VL)/VL (A21) 

Now, if SLT is the target time to meet and assuming that SLT is earlier than ETA, the amount of time that 
must be delayed in the issuing of the speed-reduction command can be estimated by 

T.TO-GO = (ETA - SLT) /ETA -GAIN = DICESP/ETA.GAIN (A22) 
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The position of the graphical marker where the speed-reduction command should be given is at a distance 
D_TO_GO ahead of the aircraft on its projected path: 

D_TO_GO = VH(T_TO_GO) - VH(DICESP)/ETA_GAIN (A23) 

The accuracy of this computation depends upon the following assumptions: 

L The deceleration time and distance from VH to VL are the same whether the deceleration is performed 
now or at a later time 

2. The ground speed remains constant during the time T .DELTA 

Although these assumptions are, in general, not true because an aircraft may change its altitude while 
holding its indicated air speed and may even change its heading (for example, from an ILS-intercept heading 
to the final- approach heading), the effects of these deviations from the assumptions are relatively small and 
do not affect the usefulness of the computation results. Furthermore, the DICE process is a self-correcting 
process. The closer the aircraft is to its speed-reduction point, the more correct are the initial values, such 
as the heading and altitude, and hence, VH and VL, and therefore, the more correct the computed ETA and 
DICESP are. 

In the final implementation of this algorithm in TIMER, the following refinements are used: 

1. With an ETA _G AIN of about 0.25 for a speed-change from 210 knots to 170 knots, any small error in the 
computation of ETA is amplified fourfold in the final value of D_TO_GO and this results in undesirable 
shifting of the graphical marker. In order to obtain a more stable ETA, a straight-line projected path to 
the runway at the final- approach heading is used regardless of whether there will be a change in heading of 
the aircraft (e.g., turning onto the localizer from an ILS-intercept heading). 

2. To prevent the controller’s display from being cluttered by a large number of graphical symbols or confusion 
as to which aircraft the advisory is intended, the speed-reduction graphical symbol of an aircraft in the 
spccd-DICE mode will not be displayed if the D_TO_GO is greater than 3.0 n.mi, 

3. A TIMER-induced speed reduction is forced upon the aircraft if it crosses the MIP at a speed higher 
than that allowed at the gate. This emulates real-world events where a pilot would either request a speed 
reduction or initiate a discretionary deceleration in preparation for executing the instrument approach. 

Since the ETA-gain is about 0.25 (for a reduction from 210 to 170 knots), there is about a 1-sec error in 
arrival time at the runway threshold for every 4 sec of error in the speed-reduction message-delivery time. This 
feature is what makes the speed- reduction advisory such a fine-tuning mechanism for delivery- time precision. 
The total amount of control available to vary the SLT, however, is fairly small. If one assumes that 5 n.mi. 
are available after the base-to-final turn before the reduction to 170 knots must begin, then the total range of 
time control is about 20 sec. (The time difference to travel a given approach distance, at a speed of 170 knots 
versus 210 knots, is about 4 sec/n.mi.) 
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Figure Al. Effect of ETA gain on 
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Time related to ILS-intercept paths 
as function of position on base leg 




Appendix B 


Definition of Interarrival Error 


B.l. Time-Based Control 

Given a time-based air traffic control process with a pair of sequential arrival aircraft with the following 
time relations: 



where 

SLTi and SLT 2 scheduled landing times at runway threshold for aircraft 1 and 2 of a pair 
ATAi and ATA 2 actual times of arrival of aircraft 1 and 2 at runway threshold 

The delivery time errors of aircraft 1 and 2 at the threshold (A^i and A£ 2 ) are defined by 


Aii = ATAi - SLTi (Bl) 

A t 2 = ATA 2 - SLT 2 (B2) 

with the delivery-time-error density distributions shown above. 

The aircraft-pair interarrival time error (IAE) is defined as 

IAE = Ai 2 - Aii ( B3 ) 

which in terms of the aircraft pair SLT’s and ATA’s is 

IAE = (ATA 2 - ATAi) - (SLT 2 - SLTi) (B4) 

IAE = (Actual separation time) — (Scheduled separation time) (B5) 


Even though each aircraft of a pair has a runway threshold time error, if the time error is the same for both 
(i.e., a constant bias), the IAE will be 0 and the pair separation will be correct. Thus, a spread of interarrival 
time errors indicates variation in desired spacing and is the attribute of interest. IAE spread is characterized 
by a statistical measure of dispersion about the mean, the variance, or its square root, the standard deviation. 

B.2. Manual Distance Separation 


Aircraft are not apriori assigned individual scheduled threshold crossing times during manual control. When 
aircraft are on instrument approaches, the controller concerns are with aircraft pair separations conforming to 
the radar separation requirements in effect on final approach. The test subject controllers were instructed to 
aim for minimum wake vortex separation. For tightly packed traffic we assume the controller is attempting 
to maximize the landing rate by keeping aircraft-pair separations to the minimum distance allowed. For this 
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case, we can treat the minimum required separation as the intended or scheduled separation. When the first 
airplane of an aircraft pair is at the runway threshold, excess separation Adi ,2 is defined by 

Adi^ = (Actual separation) — (Required separation) (B6) 

If we rewrite equation (B6) in terms of corresponding time separation, we get 

At 12 = (Time to fly actual separation) — (Time to fly scheduled separation) (B7) 

which is equivalent to equation (B5). Thus for manual data runs, an equivalent manual interarrival time IAE 7 
can be obtained. Knowing the aircraft threshold crossing times, the final-approach speed, and the wake vortex 
spacing requirement, IAE' can be calculated from 


IAE^ = At 12 = (ATA 2 — ATAi) — (Time to fly scheduled separation) 


(B8) 



Appendix C 


Single Runway Theoretical Arrival Rate 
C.l. Perfect Delivery Arrival Rate 

In general when g(x, y ) is a function of 2 random variables x and y, then the mean or expectation of g(x, y) 

is r [ 

E{g{x,y)}= / g{x,y) f{x,y)dxdy (Cl) 

Jx Jy 

with f(x , y) being the joint density function of random variables x and y. For the case where x and y are 
discrete, we can write the mean or expectation of g(x, y) as 

E{g(x,y)} = ^2^2g{xk,yn)p{xk,yn) (C2) 

k n 

where p{xj i ^y n ) is the joint probability function of x and y. 

For our case let i and j be subscripts which are random variables, each which take on values between 1 
and m where 

m number of aircraft types in traffic mix 
i lead aircraft of pair on final approach 

j trail aircraft of pair on final approach 

V aircraft speed on final 


Let us define 


= tij = Time interval between aircraft i and aircraft j when aircraft i is at 
' end of final-approach segment of length L . 


For the situation when Vj V^, the minimum required separation S^j for that aircraft pair occurs when 


aircraft i is at the threshold and is 


> vo - 


(C3a) 


For the case when Vj > Vj, the minimum required separation for that aircraft occurs when aircraft i is 
at the beginning of the final- approach segment and the separation opens until aircraft i reaches the threshold 
where 


t i 3 {V i >V j ) = ^^L --- 


1 1 


Vj Vi 


From equation (C2), wc can write the mean interarrival-time spacing {tij) as 


— E(tij) — ^ ^ t ijPij 

3=1 i = 1 

where p VJ is the probability that an aircraft pair will consist of aircraft type i followed by aircraft type j. 
For independent arrivals and first-come first-serve sequencing 

Pij ~ PiPj 
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where p l and pj are the probabilities of type i and type j aircraft being in the traffic mix. Thus equation (C4) 
can simply be rewritten as 

m tti 

tij — E{tij ) = ^ ^ ^ U jPiPj (C6) 

j=li=l 

and, for the situation when runway occupancy is not a limiting factor, the average flow rate A is 

A = T (C7) 

Hj 

Bear in mind that the separation S{j used in the calculation of tij is the exact separation required. Therefore, 
the flow rate calculated in equation (C7) is for perfect delivery precision. 

C.2. Effect of Interarrival Spacing Precision 

An operational time-based scheduling system would separate aircraft by the minimum required plus 
additional buffer separation to account for the uncertainty of aircraft delivery. If one assumes that the 
uncertainty is Gaussian with a standard deviation of cr, then we can determine the size of the average buffer 
time needed to keep the probability of separation violation or error less than some specified probability 
value P/7. For the probability of violation P £ less than 5 percent, we need a buffer time tg of 1.65cr. 

For both the overtaking (Vj > V{ ) and opening (V{ > Vj) cases of a pair of aircraft on final, figure Cl 
illustrates the total separation scheduled by a time-based system as a function of delivery error buffer and 
minimum required separation on final. The effect of interarrival spacing precision, parameterized by the 
standard deviation of the spacing uncertainty can be determined by rewriting equations (C3) as 

tij,o(Vj > Vi) = ^ + 1.65 a 

tij,a{Vi > Vj) = + L 

For the situation when runway occupancy is not a limiting factor, the resulting flow rate with a time buffer tj$ 
added to the spacing becomes 

A = -i- (C9) 


V, 


+ 1.65a 


(C8) 
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gate 


(b) Opening case when Vi> Vj. 

Figure Cl. Illustration of time-based total separation as function of delivery error buffer and minimum required 
separation for pair of aircraft on final with different speeds. 
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Appendix D 

FAS A Learning Effect on Number of Vectors Issued Per Aircraft 

Debriefing discussions and controller comments suggested a learning or training effect after operating with 
the graphic marker and DICE countdown formats. After using the direct automation aids (graphic and DICE 
formats), there seemed to be a tendency, among those subjects who used intermediate vectors (cutting corners), 
toward using more squared downwind/base vectors (classic trombone pattern) in subsequent manual and 
centerline slot marker runs. The manual format was always the first format tested in each pattern-speed 
procedure. 

The purpose of this appendix is to determine whether the vectors per aircraft conclusions (section 5.2) were 
voided if this learning effect existed. We assert that the comparisons and trends were correct and if anything 
the “true” mean vectors per aircraft for the manual and centerline slot marker formats were somewhat higher 
than those shown in figure 20. To support this assertion, the data in figure 20(a) were separated by subject 
controllers performing the 170-knot procedure first and those performing the 210-knot procedure first, as shown 
in table Dl. 

These data appear to support the existence of a change in the vectors per aircraft after using the automated 
direct aids particularly for the manual format. We used a simple linear model based on an observed condition 
and an assumption to estimate the automation learning influence on the manual format . The observed condition 
was that there was an inherent difference in the mean vectors per aircraft between the 210-knot and the 170-knot 
speed-procedure for the manual format. The assumption was that the learning effect for the graphic and DICE 
formats was the same whether the subject controller was exposed first to the 170-knot speed procedure or the 
210-knot speed procedure. An analysis using this assumption and a simple linear model is as follows: 

Vlm = Mean learning effect on manual format vectors per aircraft 

Frm = Mean reduction in vectors per aircraft in going from 170- to 210-knot pattern-speed procedure 

for manual format j 

V 170M/170 1st = Mean vectors per aircraft of 170-knot manual format when 170-knot procedure was first 

— i 

V 170 M /170 2nd — Mean vectors per aircraft of 170-knot manual format when 170-knot procedure was j 

second 

1^210^1/210 1st — Mean vectors per aircraft of 210-knot manual format when 210-knot procedure was first j 

^210M/210 2nd — Mean vectors per aircraft of 210-knot manual format when 210-knot procedure was | 

second ! 

The linear model is written as ; 

^170M/170 1st “ V LM - V RM = ^210M/2102nd 
r 210M/210 1st - V LM + V RM = V 170M/I70 2nd 

Substituting values from table Dl gives ‘ 

2-92 — F lm — F rm = 2.11 | 

2-46 -F LM +F RM = 2.38 | 

Solving these simultaneous equations gives 

F l M = 0.45 vector/aircraft * 

Two ways exist to estimate the vectors per aircraft for the_l 70-knot procedure manual format prior to any 5 

learning effect. One is to use, as our estimate, the value of V 170 M /170 1st given in table Dl(a), which is 

V 170 M/ 170 1st = 2.92 vectors/aircraft 
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With the use of the value of ^i 70 M/i702nd given in table Dl(b), the other estimate is calculated from 

V i 7qm / 1 70 2nd + Vlm = 2.38 + 0.45 = 2.83 vectors/aircraft 

We used as our best estimate of the 170-knot procedure, manual vectors per aircraft, prior to any graphic or 
DICE format learning effect the average of the above two estimates, which is 

V\ 7qm = 0.5(2.83 + 2.92) = 2.88 vectors/aircraft 

Similarly, there are two ways to estimate the vectors per aircraft for the 210-knot procedure manual format 
prior to any learning effect. One is to use the value of k r 2 ioM/210 1st gi yen in table Dl(b), which is 

V 210M/210 1st = 2-46 vectors/aircraft 

With the use of the value of U 2 iom/ 210 2nd given in table Dl(a), the other estimate is calculated from 

^210M/2102nd + ^LM = 2.11 + 0.45 = 2.56 vectors/aircraft 

Our best estimate (V^lONl) °f the vectors per aircraft for the 210-knot procedure manual format prior to any 
graphic or DICE format learning effect, is the average of the above two 210-knot estimates, which is 

V 21 OM — 0.5(2.56 + 2.46) — 2.51 vectors/aircraft 

Unlike the manual format, which was always first, the test position of the centerline slot marker in the 
first pattern-speed procedure series varied from subject to subject. Sometimes the centerline slot marker was 
performed immediately after the manual format and before subject exposure to the graphic or DICE formats. 
Other times the centerline slot marker was performed after both graphic and DICE formats and the subject 
was exposed to a possible full learning effect. There was of course the intermediate case where the centerline 
slot marker was performed between that of the graphic and the DICE formats. As a consequence, obtaining 
an estimate for the centerline slot marker vectors per aircraft, prior to a learning effect, was not as direct as 
for the manual format . 

Two assumptions were made in order to estimate the mean vectors per aircraft for the centerline slot marker 
prior to a learning effect. The first assumption was that most of the learning effect had occurred during the 
practice and data runs of both the graphic marker and DICE countdown formats in the first speed procedure 
series and before starting the second speed procedure series. The second assumption was that the inherent 
difference in mean vectors per aircraft between the manual format and the centerline slot marker format is not 
changed by the learning effect. Using these assumptions, the difference between the mean vectors per aircraft 
for the manual format (Ui70M/i7O2nd) an <4 the mean vectors per aircraft for the centerline slot marker format 

(^l70C/1702nd) a fter the learning effect has occurred is 

^170M/1702nd ~ ^170C/1702nd = 2-38 - 1.99 = 0.39 vector/aircraft 

Using our best estimate of the vectors per aircraft for the manual format 170-knot procedure (Ui70m)? the 
estimated vectors per aircraft for the 170-knot centerline slot marker format (Vi 7 oc) 5 prior to any graphic or 
DICE effect, was 

^170C = ^170M - (^170M/170 2nd ~ ^170C/1702nd) 

= 2.88 - 0.39 = 2.49 vectors/aircraft 

Similarly, the difference between the manual mean vectors per aircraft (U 2 ioM/2102nd) an< ^ the centerline slot 
marker vectors per aircraft (V 210 C/ 210 2nd) > a fter the learning effect has occurred, is 

V 21 OM/ 21 O 2nd - V 210 C /210 2nd = 2.11 - 1.70 = 0.41 vector/aircraft 
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Again using our best estimate of the vectors per aircraft for the manual format, 210-knot procedure (V^iom)i 
the estimated vectors per aircraft for the 210-knot centerline slot marker format (V 210 C)? pri° r to any graphic 
or DICE effect, was 

^2 IOC = ^210M “ (^210M/2102nd “ ^210C/2102nd) 

= 2.51 - 0.41 — 2.10 vectors/aircraft 

The result of this analysis supports the assertion that the subject controllers did change the mean number 
of vectors per aircraft issued, for both the manual and centerline slot marker format, after being exposed to 
the direct automation aid graphic marker and DICE countdown formats. This change was a mean reduction 
of about 0.4 vector per aircraft for the manual format and about 0.3 vector per aircraft for the centerline 
slot marker format. In addition, the analysis reinforced that all earlier mean trend comparisons were valid 
(section 5.2), particularly that the graphic marker and DICE countdown formats had significantly fewer mean 
vectors per aircraft than either the manual or centerline slot marker formats. 


Table Dl. Mean Vectors Per Aircraft for Display Formats 

" Subjects are divided according to testing order of speed procedures. To preserve anonymity, 
controller subject numbers have been randomized and bear no relationship to original 
test order appearance; however, listed subject numbers are consistent for all report data. 


(a) Subjects performing 170-knot procedure first 



Mean vectors per aircraft 



170-knot procedure first 


210-knot procedure second 

Controller 









test subject 

MAN 

GM 

DICE 

CSM 

MAN 

GM 

DICE 

CSM 

2 

2.571 

1.400 

1.588 

2.000 

2.146 

1.500 

1.500 

1.755 

5 

3.116 

1.489 

1.347 

2.556 

2.408 

1.429 

1.510 

1.804 

8 

3.065 

1.510 

1.429 

3.267 

2.727 

1.565 

1.510 

1.250 

9 

1.958 

1.489 

1.667 

1.717 

1.479 

1.429 

1.404 

1.457 

10 

3.756 

1.596 

1.532 

2.375 

2.400 

1.565 

1.580 

2.022 

11 

3.381 

1.490 

1.578 

2.085 

1.980 

1.711 

1.563 

1.956 

12 

2.565 

1.451 

1.429 

2.146 

1.620 

1.388 

1.451 

1.633 

Mean 

2.916 

1.489 

1.510 

2.307 

2.109 

1.512 

1.503 

1.697 

St dev 

0.598 

0.060 

0.112 

0.501 

0.449 

0.111 

0.061 

0.274 


(b) Subjects performing 210-knot procedure first 



Mean vectors per aircraft 


170-knot procedure second 

210-knot procedure first 

Controller 









test subject 

MAN 

GM 

DICE 

CSM 

MAN 

GM 

DICE 

CSM 

1 

2.224 

1.565 

1.659 

1.933 

2.580 

1.592 

1.630 

1.729 

3 

2.143 

1.360 

1.449 

1.932 

2.327 

1.667 

1.420 

1.898 

4 

2.731 

1.816 

2.065 

2.356 

2.843 

1.565 

1.520 

2.490 

6 

2.362 

1.500 

1.526 

2.089 

2.404 

1.429 

1.426 

1.938 

7 

2.432 

1.612 

1.531 

1.638 

2.136 

1.383 

1.540 

1.714 

Mean 

2.378 

1.571 

1.646 

1.990 

2.458 

1.527 

1.507 

1.954 

St dev 

0.227 

0.167 

0.246 

0.262 

0.268 

0.118 

0.087 

0.316 
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Appendix E 


Lookpoint Data Recording, Development, and Analysis 

E.l. Facility Description and Recording Methodology 

The oculometer facility computes and stores a time history of eye-scanning events. The block diagram 
(fig. El) includes the components of the system. The oculometer projects a collimated near-infrared beam of 
light onto the subject’s eye. The system depends on algorithms that can compute a lookpoint given the relative 
position of two reflections from a single eye of the subject. The computer compares the large back lighted pupil 
reflection to the much smaller and more intense corneal reflection. By using split image techniques, the system 
directs the illuminating beam through the same tracking mirror system (block 4) that collects the reflected 
images. It uses the angles of the two automatic tracking mirrors and the manually controlled focus of the 
eye-camera optics to correct the lookpoint calculation for subject head position. The oculometer electro-optic 
head (blocks 4 and 5) is located directly in front of the subject (block 8) and just below the simulated radar 
display. This location is well outside the final controller’s normal scan area, which is concentrated around 
the center of the display. The subject can detect only a dull red light in the head’s mirror system. For 
the purposes of calibration and monitoring real-time performance, the system mixes (block 9) the computed 
lookpoint position with the PPI video signal that is nonobtrusively recorded (block 11) from a repeater display 
(block 10), also shown in figure E2. The resulting combined display (fig. El, block 12) contains a small circle 
of light representing the lookpoint as it moves among the display symbols. An observer can monitor in real 
time both system and subject performance by viewing this combined signal. A video recorder (block 13) stores 
this signal on tape for postrun analysis. 

Most of the components of the oculometer system are located in the Human Engineering Methods (HEM) 
laboratory one floor below the TRACON simulation facility. Figure E3 is a photograph taken in the HEM 
laboratory. In the background corner one can see a display monitor that has the mixed video with the controller 
PPI display and the controller’s lookpoint superimposed. The three 5-in. video monitors in front of the main 
system operator (in the foreground) are used for system monitoring and control. (Details of these displays 
do not show well in the photograph.) The left monitor is a duplicate of the mixed PPI/lookpoint display. 
The central monitor shows the bright corneal reflection on the much larger and darker pupil reflection in the 
background. The right monitor shows the subject’s entire head. The operator uses this camera to observe 
the subject and as an aid to recapture the subject’s eye after losing track. The signal on the cathode-ray 
tube above the monitors is the sweep from the eye camera used to determine the relative position of the two 
reflections. The central narrow peak indicates the corneal reflection. The broader peak at about half-voltage 
represents the pupil reflection and the low voltage baseline represents the rest of the eye. By keeping track of 
video sweep count and timing when voltages cross specified levels, the system determines the center of each of 
the two reflections and thus their relative position on the camera vidicon. 

Figure E3 does not include the oculometer computer. However, it does show several of the digital readouts 
and control inputs for the oculometer computer such as potentiometers, a standard typewriter keyboard, and 
a joystick (under the operator’s right hand). Their principal use is for prerun calibration, but the operator also 
uses them to dynamically compensate for the subject’s posture adjustments. The mirror tracking is automatic 
and works well. The joy stick is a manual augmentation for the mirror tracking. The operator uses it to 
override the search algorithm when the eye is out of track and the system is trying to reacquire. He does 
not use the joy stick often but when used it speeds up reacquisition considerably. The operator controls the 
mirrors, the eye camera focus, and collimated infrared beam intensity. The operator also determines and enters 
system parameters during calibration. These include parameters to adjust for intersubject differences in corneal 
curvature. The microcomputer in the background of figure E3 collects and stores the visual events in real time 
in its random access memory. At the end of each run, the operator copies the records to a disk file for long-term 
storage. Each record spans a variable time duration that is an integer multiple of the oculometer sample period 
(30 samples/sec). There are four data fields per record containing lookpoint coordinates (2 dimensions), pupil 
diameter, and time duration of event. For an out-of- track event, the system records lookpoint coordinates as 
zero and stores a status code in the pupil diameter field. Once per rotation of the simulated TRACON radar, 
that is, every fourth second, the system stores one other field on a second file. This field contains the sequence 
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number of the visual event last completed as the ATC simulation radar frame started. The data reduction 
algorithms use this information later for synchronizing the recorded simulation data with the oculometer data, 

E.2. Data Reduction 

The eye- scanning event file and the PPI/oculometer synchronizing file were both recorded in the oculometer 
facility as discussed in section E.l. In addition, the ATC simulation simultaneously wrote a third real time 
file. This third file contained the state of the display including the position of aircraft, the tag information, 
and the state of any currently active graphic aids. The simulation computer updated the display and saved 
one of these records each radar sweep. Postprocessing synchronized the three files, that is, assigned each 
fixation to its corresponding radar sweep. Then, a filtering process reduced the oculometer data. It removed 
a few known bad records (less than 0.003 percent) and some of the short (noise) out-of-track records. In 
addition, the process combined adjacent records that pertain to the same fixation into a single record with a 
longer duration. This pattern occurred, for example, when a noise record or blink interrupted a fixation. A 
separate report (ref. 22) documents in detail the data reduction process for applying an oculometer to an ATC 
controller-display evaluation. The report also describes the details of the computer programs developed for the 
collection, reduction, and analysis of the data. 

E.3. Display Object Identification 

After filtering, the algorithms assigned each member of the filtered set of fixations to a display object 
by searching the corresponding radar sweep data. If they found no display object near the lookpoint, they 
classified the fixation display object as unknown. In order for a lookpoint to be coupled with a display object 
the distance between them had to be less than 0.57 in. on the screen or just over 1 n.mi. for the display scale 
used in the experiment. In most cases, if two display objects were within this distance of the lookpoint, the 
algorithm assigned the fixation to the closer of the two. The exception was when an aircraft was very close to 
its graphic turn marker, graphic speed marker, or centerline slot marker. In those cases, the lookpoint, when 
within 0.57 in. of both the aircraft and the FASA, was assigned to an appropriate combination category of 
special interest in this study, 

E.4. Resulting Files 

The “merge” files resulting from this postprocessing data reduction and display-object identification process 
consisted of both in-track and out-of-track records, one file per run. The merge file fixation records contain 
the coordinates of both the lookpoint and the display object, as well as the distance between. They also 
contain information on the state of the FASA. For example, if the controller was looking at an aircraft during 
a DICE run, the record would contain the countdown and heading information but only if the display provided 
that information during that particular radar update. The following are means from the 24 (12 controllers at 
2 approach speeds) manual runs: 

The mean number of records per run was 5648 after filtering 

The mean in-track time was about 85 percent of total test time 

The time for unknown display object fixations was about 8 percent of in-track time 

The mean time duration for all fixations was 0.68 sec 
These merge files are the basis for the lookpoint analysis presented in the main text. 

E.5. Analysis 

These merge files contain large amounts of data, typically over 600000 bytes per file. There is a total of 
96 files excluding practice runs (12 controllers using 8 test conditions). These data were used to address two 
general types of questions. The first type of questions concerns the scan pattern of the controllers: Where 
are they looking and at what, and how do they budget their fixation time over types of display objects or 
over areas (zones) of the display? This would be very difficult to determine objectively or accurately without 
an oculometer. The second type of question concerns the existence and identification of parameters in the 
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scan pattern that are statistically sensitive to changes in the display. After establishing a correlation between 
these changes in scan patterns and the display changes, the next step was to evaluate the implications of these 
changes with respect to the merits of particular display characteristics. 

E.6. Types of Statistics 

In this study, three general types of statistics were examined: 

1. Mean fixation time. 

2. Percentages such as “in-track time as a percent of total time” or “time spent looking at aircraft as percent 
of total in-track time.” 

3. Number of cross-check scans (CCS) and corresponding time spent as a percentage of total in-track time. A 
CCS is defined as a succession of fixations on a pair of display objects. It represents an ordered scan pattern 
which is alternating between the two objects of the pair, uninterrupted by a third display object or long 
out-of-track time. For a pair of display objects A and B, the lookpoint sequence A-B is counted as a CCS of 
order 2, the sequence A-B-A is of order 3, et cetera. The occurrence of high-order CCS’s is considered bad 
because it indicates increased controller concern or information gathering. In this study, CCS’s of order 2 
to 4 and (5 or greater) were tallied. The cross-check scan measure is potentially a significant concept which 
has not been formally documented by its originator, Randall L. Harris of the NASA Langley Research 
Center. 

These 3 types of statistics were accumulated for each display object and display object pair as well as for 
display zones. Four zones in the terminal area were defined: downwind, base, final, and “everywhere else.” 
The zones are depicted in figure E4. A further breakdown of the statistics was based on the occurrence of an 
event such as “while speed marker was on” or “while DICE was showing in data block.” 
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Figure El. Operational block diagram of oculometcr system interaction with ATC simulation. 
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Figure E2. Repeater display of final controller’s PPI which was recorded for video mixing with oculometer 
lookpoint coordinates. 
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Figure E3. Oculometer control and recording facility. 
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Figure E4. Illustration of zones defined on final controller’s display for Iookpoint analysis. Zone 4 is everywhere 
on display not defined by zones 1, 2, and 3. 


Appendix F 


Controller Questionnaires and Responses 
F.l. General Description 

This appendix includes all questionnaires and responses (evaluations and comments) of the questionnaires 
except the graphed results included in the main text and the comments from the verbal debriefing session. 
In the interest of conserving space, instructions for completing each type of questionnaire are included only 
once, and the print size has been reduced from that used in the original questionnaire. Types of questions and 
sections they appear in arc as follows: 


Specific Format Questionnaires Combined Subject Evaluations and Comments F.2 

Format Questionnaires for 170-Knot Pattern-Speed Procedure F.2.1 

Manual Format F.2.1. 1 

Graphic Marker Format F.2. 1.2 

DICE Countdown Format F.2. 1.3 

Centerline Slot Marker Format F.2. 1.4 

Format Questionnaires for 210-Knot Pattern-Speed Procedure F.2. 2 

Manual Format F.2. 2.1 

Graphic Marker Format F.2. 2. 2 

DICE Countdown Format F.2. 2. 3 

Centerline Slot Marker Format . . . , F.2. 2. 4 

Task Load Index — Individual Subject Controller Results F.3 

TLX Description • • F.3.1 

Individual Subject Controller Results F.3. 2 

Rating and Ordering Test Formats Questionnaire Only F.4 

Comparison of Format Questionnaires for 170- and 210-Knot Pattern-Speed Procedures — Subject 

Evaluations and Comments F.5 

Final Debriefing Questionnaire — Subject Evaluations and Comments F.6 


Researcher Topic Guide for Final Verbal Debriefing F.7 


F.2. Specific Format Questionnaires — Combined Subject Evaluations and Comments 

This section contains the results of the format questions which were administered at the completion of each 
format data run. Each question is provided along with the number of responses for each of the five positions 
along the graphic scale. All subject comments are also included. 

F.2.1 Format Questionnaires for 170-Knot Pattern-Speed Procedure 


F.2.1. 1 Manual Format for 170-Knot Procedure 

Note: For each experimental condition, the specific- for mat questionnaire was headed by a set of instructions 
(such as shown below). However for report purposes, these general instructions are not included in all 
subsequent corresponding questionnaires requiring a simple response to a scale defined by end-point descriptors. 



SUBJECT CONTROLLER QUESTIONNAIRE 


Condition - Manual Control (170 kts Final Turn) 

Instruction: Carefully read each of the following statements, then on the associated scale enter an “X” in the box which 
most closely represents your position, feeling, or judgment concerning the statement. Each scale has two endpoint descriptors 
that define the scale. For example: 


The representation of a TRACON operating under IFR conditions in the test environment was 


unrealistic 


1 

CO 

CM 

4 

5 


realistic 


Marking the box labeled “1" indicates you feel the test environment was seriously unrealistic. Box “2” indicates the simulation 
was primarily unrealistic with a few realistic features. Box “3" indicates roughly equal number of realistic and unrealistic features. 
Box “4” indicates the simulation was primarily realistic with minor shortcomings. Box “5” indicates the test environment was 
realistic in representing an operational IFR TRACON condition. 

There is room for additional comments below each scale. Any comments which clarify or explain your view will be helpful 
information for our evaluation of controller opinion. 

1 . The simulation of a TRACON operating in IFR conditions (single runway configuration) during a moderate to busy 
traffic period was... 


unrealistic 


0 

1 1 

9 

1 


realistic 


Comments 

• My only comment was separation on short final would normally be adjusted by tower for the final controller. 

• No variable performance aircraft had to be mixed. 

• Once the tower had the aircraft they could issue advisories on speed and even issue speed reductions to insure 
separation. 

2. Given the organization of traffic performed by the feeder and automation in this session, the landing sequence 
was... 


not obvious 


0 

0 

0 

2 10 


readily apparent 


Comments 

• Feeder gave excellent spacing, but not too much interval between traffic. 

3. In today’s manual TRACON environment there is no automation to aid the feeder in organizing traffic for the final 
controller. Given the aircraft spacing and organization performed by the feeder/automation interaction In this 
session, the effort required to set up the landing sequence as compared to today’s manual environment was... 


greatly increased 


greatly reduced 


Comments 

None 


4. Based on your experience with aircraft performance in IMC, the simulated aircraft flight paths and maneuvers 
were...(Please identify any deficiencies.) 


unrealistic 


0 

1 

4 

5 

2 


realistic 


Comments 

• Due to the high altitude I felt the aircraft turns to final were somewhat slower than normal. 

• Real pilots tend to miss more instructions. 

• Aircraft inside the marker seemed to “DIE”. 

• I think the speed would change too much between 7 miles and 4 miles on final. 
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5. Communication with pseudo pilots was...(Please identify any deficiencies.) 


unrealistic 


0 

1 

CM 

2 

7 


realistic 


Comments 

• Compared to the real world the pseudo pilots here are very organized in their transmissions. In real life blocked 
transmissions block out everything. 

• About as good as simulation can be. 

• A few times the voice disguiser made transmissions difficult to understand. 


6. The Interaction and coordination with the feeder controller were...(PIea$e identify any deficiencies.) 


unrealistic 


0 

1 

3 

6 

2 


realistic 


Comments 

• Would be easier with an interphone/intercom button. 

• No landlines, buttons to press, other traffic diverting his attention. 

7. The suitability of the simulated radar display to perform the required control task was.. .(Please identify any 
deficiencies.) 


seriously deficient 


10 


adequate 


Comments 

• Unable to adjust to personal specs. 

• It is adequate, but it obviously lacks the primary target and beacons control slash. 

8. The initial briefing and training on Denver airspace/procedures and the simulation interface necessary to do a 
representative job of controlling traffic was.. .(Please identify any deficiencies.) 


seriously deficient 


adequate 


Comments 

• I would like to look at the ILS APP. charts and if you have them, the profile descent into Den. 


9. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was.. .(if not normal, please indicate reason.) 


below normal 



above normal 


Comments ^ - 

• I still am feeling my way around the airspace configuration and simulator characteristics. 

• Overall concerned with trying to maintain proper position for oculometer. 

10. Additional comments on performance or conditions, if any. 

• Very enjoyable and educational so far. 

• I felt a little helpless once the a/c were cleared for the approach because I couldn’t have strung them out on the 
downwind or separation without impacting the feeder. 


F.2.1.2. Graphic Marker Format for 170-Knot Procedure 

1. Adapting to the use of the graphic turn advisor was... 


difficult 


0 

0 

0 0 

12 


easy 
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Comments 


• Extremely valuable automation aid, it helped me be much more aware of all aspects of airspace and separation involved. 

2. The turn positions indicated by the graphic turn advisor, as compared to the positions you would have turned 
aircraft for spacing on final... 


strongly differed 



closely agreed 


Comments 

• They were probably more accurate then my own. 

3. The tabular sequence list presented on the right of the display was... (If useful, how was it used? 


useless 


7 

1 

2 

2 

0 


very useful 


Comments 

• I never used the list once. 

• Once again, I didn’t use it. 

• Checked sequence between ties. 

• Referred to it occasionally. 

• Hard for me to read - actually alpha numerics are on the whole smaller than I prefer. 


4. Indicate how the graphic turn advisories (170 kts procedure) affected your workload , as a final controller, as 
compared to manual operation (170 kts procedure) with no computer aid. 


increased 


0 

1 

0 

5 

6 


reduced 


Comments 

• Greatly enhanced my ability to insure separation and give more quality turn-on’s. 

• Fewer decisions had to be made. 


5. Indicate how the graphic turn advisories (170 kts procedure) affected your spacing on final approach as compared 
to manual operation (170 kts procedure) with no computer aid. 


less precise 


0 

1 

1 5 

5 


more precise 


Comments 

• There was less guess work and more evident or obvious spacing, and I was able to adjust spacing easier. 

• The computer was more accurate. 

6. In terms of display clutter, the addition of the graphic turn marker symbol to the display generated... 


excessive clutter 


0 

0 

4 

in 

CO 


no perceptible clutter 


Comments 

• No real distraction in this environment. 


7. Relative to the point where aircraft were to be turned, the graphic turn advisories appeared ahead of the aircraft 
at a distance that was... 


0 


0 


12 


0 


0 


too far 


too close 









Comments 


• Very acceptable distance. 

• About right. 

• Most of the time, the turns were good, but a few times the turns were off. The computer needs too much time to 
recalculate when strange things happen. 


8. Indicate how your focus on the aircraft/graphic-turn-marker position relationship affected your attention to alrcraft- 
to-aircraft spacing. 


strongly reduced 


1 

3 

1 

4 

3 


not affected 


Comments 

• I was very confident in the automation and it never really hampered my attention to spacing. 

• My scan was reduced, however, I feel the graphic reduced my scan more. 


9. Indicate how your focus on the aircraft/graphic-turn-marker position relationship affected your attention to the 
overall traffic picture. 


strongly reduced 


not affected 


Comments 

• I was more tuned to the final but overall no change. 

• As I became comfortable with the system my scan was not affected. Initially I felt I watched the markers longer. 

10. Having a numerical heading advisory along with the displayed graphic turn advisor would be... 


undesirable 


4 

1 3 

2 

CM 


desirable 


Comments 

• It wouldn’t change the fact that under normal conditions a controller is going to adjust to what suits him/her regardless. 

• I think unnecessary would be a better word. 

11. Having the graphic turn advisor displayed until aircraft completed their turn (to monitor turn performance) resulted 
in... 


excessive clutter 


1 

3 

1 

2 5 


no perceptible clutter 


Comments 

• Again, in this environment no clutter at all. 

• Once the turn has been issued the marker has served its purpose and no longer needs to be displayed. 

• Occasionally I would have to look at the graphic turn advisor to see which aircraft it was meant for. 


12. Rather than the graphic advisor made up of the three line segments, as used in the test, a different form of graphic 

turn advisor format would be.. .(Please indicate your preference if any. Some examples include a turn arc ( ^ ), 

a symbol made up of two directed line segments ( I ), or a simple symbol at the position along the path where 
the turn instruction is to be issued.) 


not preferable 


4 

2 

3 

2 

1 


preferable 


Comments 

• The line segments are more distinguishable, therefore, they were more useful. 

• No preference. 
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• This seemed fine, but an arc may have worked well also. 

• ARC - aircraft do not make 90 degree turns. 

13. Relative to the final controller s job, indicate your reaction and feeling to having a computer suggest where aircraft 
should be turned for spacing. 


undesirable 


1 


6 


desirable 


Comments 

• A very useful and stress reducing tool, I would work with it tomorrow. 

• Personally I would not enjoy the job the way I do today. Working airplanes is fun, monitoring a CRT is boring. 

• The computer is too rigid and takes too much time for recalculation. 

14. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was. ..(If not normal, please Indicate reason.) 


below normal 


0 

0 

8 

3 

1 


above normal 


Comments 

• I was curious to see my reaction to the automation, and was more relaxed and attentive with it, versus manual control. 

15. Additional comments on performance or conditions, if any. 

Comments 

• This was the easiest to work with of the aids for me. 

• In some cases it appeared that aircraft did not perform as expected when within FAF. 

• I felt very comfortable with this format. 


F.2.1.3. DICE Countdown Format for 170-Knot Procedure 


1. Adapting to the use of the DICE turn advisor was... 


difficult 


easy 


Comments 

• More data block information than I’m used to working with, but it didn’t deter my control. 

• Felt that I could initiate my own turns - instead, I waited for prompts. 

2. The times-to-turn Indicated by the DICE turn advisor, as compared to* the times you would have turned aircraft for 
spacing on final... 


strongly differed 


1 

2 

0 

7 

2 


closely agreed 


Comments 

• When I could accurately catch the turns they were very accurate, however, if it indicated I would be through the localizer 
I went with my own turns entirely with what I feel were better results. 

• My tendency is to get aircraft established on final approach course further out, but this was good to see. 

• When you had a situation out of the ordinary or an aircraft got extended, the computer did not give updated headings. 

3. The tabular sequence list presented on the right of the display was...(lf useful, how was it used?) 


useless 


5 


2 


3 


1 


very useful 


77 








Comments 


• I never used it. 

• I don’t believe 1 referred to the tab list once. 

• Detracted somewhat - it distracted my attention from the aircraft. 

• It is annoying and distracting. It is another thing to look at. If you change the sequence, the tab sequence will confuse 
you. 


4. indicate how the DICE turn advisories (170 kts procedure) affected your workload, as a final controller, as compared 
to manual operation (170 kts procedure) with no computer aid. 


increased 


1 

1 

3 

3 

4 


reduced 


Comments 

• Depending on the situation, if I had no distractions the turn advisor was a great tool. However, if I would have to 
resolve a conflict in normal conditions away from final, it really didn’t change my control of the situation, I’d tend to go 
manually. 

• Reduced my workload a great deal. 

• I was sometimes concentrating too much on the prompts and less on the separation. 

5. Indicate how the DICE turn advisories (170 kts procedure) affected your spacing on final approach as compared 
to manual operation (170 kts procedure) with no computer aid. 


less precise 


more precise 


Comments 

• With the advent of this software, my spacing was probably much better than manually. 


6. In terms of display clutter, the DICE countdown and suggested heading value which are added to the data block, 
generated... 


excessive clutter 


0 


5 


no perceptible clutter 


Comments 

• Again in this environment it was no distraction. I personally do not prefer a lot of writing in the data block. 

7. Relative to the time when aircraft were to be turned, the DICE time-to-turn advisories appeared in the data block 
at a time that was... 


to early 


0 , 

0 

12 

0 

0 


too late 


Comments 

• 1 leaned toward my own headings when a/c were late to turn on, however, when there was no late turn, the advisory 
was perfect. 

• Adequate. 

• Hard to tell - sometimes, I felt like I was turning at exactly the right time, but it seemed too late. 

• Sometimes the countdown jumps too quick and it is tough to keep an eye on it. You end up focusing on this countdown 
instead of scanning the whole scope. 


8. Indicate how your focus on the DICE countdown value affected your attention to alrcraft-to-aircraft spacing. 


strongly reduced 


_Jj 

4 

1 

5 

1 


not affected 


Comments 

• Aside from this simulation, if I had to closely monitor final accuracy spacing, I would not have been able to use the 
Dice as effectively. 
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• My scan was reduced to ensure I turned the a/c at the appropriate time. 

• initially, focus on the countdown rate seemed to take precedence over focus on actual spacing. 

• At first I was double-checking but it seemed to be providing good spacing - so I began to rely more on it and less on 
myself. 


9. Indicate how your focus on the DICE countdown value affected your attention to the overall traffic picture. 


strongly reduced 


0 


5 


not affected 


Comments 

• I tended to concentrate on getting the Dice advisor to do the work and at times it consumed most of my attention. 

• The Dice offers the controller more time to become complacent. 

• Distracted attention from overall picture. Less thinking and planning on my part involved. 


10. Generally the base and final turn headings are predictable and constant. As an aid to the final controller, do you 
feel the numerical heading advisories in the data tag were... 


unnecessary 


necessary 


Comments 

• I could see in bad weather this software would be a great assistance but in normal conditions I fee! the controller will 
trust his own headings. 

• It’s my feeling that depending on the a/c position from the airport and localizer I could make the heading assignment. 

• In the test environment it is needed but after working with the system, It could possibly be inhibited. 

• The headings are nice to have. 

• Probably not necessary but were helpful and utilized. 


11. Rather than the DICE countdown format, as used in the test, a different form of time information In the data block 
for suggesting the time-to-turn would be. ..(Please indicate any suggestions for improving the display format.) 


not preferable 


5 

1 

4 

2 

0 


preferable 


Comments 

• A slower type of countdown system. Instead of letting it go from 75 to 64 to 53 to 28, etc. have it start at 7, 6, 5, 4,3, etc. 

• Graphic position symbols. 


12. Relative to the final controller’s job, indicate your reaction and feeling to having a computer suggest when a 
aircraft should be turned for spacing. 


undesirable 


1 

0 

CM 

6 


JJ 


desirable 


Comments 

• I feel if the software is a viable concept and appeals to all I think it is greatly needed. 

• While I was running the problem I tended to get a little bored. The Dice makes the job very easy and takes away from 
my own personal satisfaction. 

• It’s a little tedious, but certainly reduces anxiety levels. Also have a tendency not to concentrate as hard. 

• Would need more time to evaluate - maybe I’d get used to it and choose on a case by case basis when it could be 
valuable. 

• It’s desirable when you agree 100% with the computer evaluation. If you disagree, the computer does not seem to be 
that much of a help. 
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13. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was.. .(If not normal, please indicate reason.) 


below normal 


° 

3 

6 

2 

1 


above normal 


Comments 

• I found that because of the precise nature of this aid I had to be much more aware of my tasks. 

• My alertness would compare to a slow session in the manual mode. 

• It became a little tedious, but I don’t believe in the real world it would be quite so, due to more human and uncontrollable 
factors. 

• It took me out of the decision-making process quite a lot - thereby, I was responding to more than initiating the traffic 
flow. 

14. Additional comments on performance or conditions, if any. 

None. 

F.2.1.4. Centerline Slot Marker Format for 170-Knot Procedure 


1. Adapting to the use of the centerline slot marker was... 


difficult 


easy 


Comments 

• Extremely hard process to make the scheduled “circle", I felt a little more adapted after the second run but not confident. 


2. The sequence indicated by the centerline slot marker, as compared to the sequence you would have chosen 
without the slot markers... 


strongly differed 


0 

2 

1 

5 

4 


closely agreed 


Comments 

• A couple of times, aircraft were inbound that I would have changed sequence on immediately, contrary to what the 
centerline marker was instructing me to do. 

• On two occasions I would have changed sequence. 

• Except for one situation - I think that the sequence was what I'd have used. 

• I would have changed the sequence 3 or 4 times. 


3. Indicate how the centerline slot marker (170 kts procedure) affected your workload, as a final controller, as 
compared to manual operation (170 kts procedure) with no computer aid. 


increased 


3 

4 

2 

2 

1 


reduced 


Comments 

• Greatly increased my workload by making me concentrate on final and spacing close in. 

• Had to adjust to using it - after repetitive usage it would become more natural. 

• The slot marker posed more of a problem at the 170 kt procedure. 

• Probably would decrease workload when l become more used to working with it. 


4. In the normal case (when slot markers were not noticeably shifted to adjust for spacing errors) vectoring aircraft 
into their slot markers was... 


difficult 


5 

1 

6 

0 


easy 
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Comments 

• I had a hard time hitting the slot markers on a consistent basis. This of course could be different given more exposure 
to it. 

• Not easy, but manageable. I sometimes had to use additional vectors. 


5. Indicate how often an extra effort was made to precisely center the aircraft in its slot marker. 


never 


0 

1 

1 

8 

2 


always 


Comments 

• Often I had to adjust headings and speeds to compensate for centering. 

• It was more difficult when the slot marker kept jumping backward because it thought the front aircraft would not hit the 
slot marker. 


6. In terms of display clutter, the slot markers displayed on the extended runway centerline generated... 


excessive clutter 


no perceptible clutter 


Comments 

• No more or less than the other automation system. 


7. Indicate how your focus on the aircraft/slot-marker-position relationship affected your attention to aircraft-to* 
aircraft spacing. 


strongly reduced 


2 


5 


3 1 


not affected 


Comments 

• I concentrated much more on aircraft spacing than any other aspect this session. 

• I was less likely to scan A/C inside the FAF. 

• Somewhat reduced since my concentration was shifted to the slot rather than the aircraft. 


8. Indicate how your focus on the aircraft/slot-marker-position relationship affected your attention to the overall 
traffic picture. 


strongly reduced 


1 

3 

3 

5 

0 


not affected 


Comments 

• I was paying way too much attention to centering aircraft in their slots and spacing involved. 

• I found I spent more time fine tuning my sequence to hit the marker. Without a marker I am much easier on myself if 
I miss a hole by 1/4 mile. 

• I just assumed the slot marker would hold separation down to the runway. In two cases it did not. 


9. In spacing/sequencing aircraft on final, to what degree do you feel the slot markers provide useful information. (If 
useful, how?) 


not useful 


0 

1 

2 

5 

4 


useful 


Comments 

• My only feeling is it prompted me to turn aircraft to base much more expeditiously. 

• Once a/c are on final, a marker could be used to make the separation, however, vectoring to intercept at the slot marker 
I found too demanding. 

• Help to focus on the position the aircraft should be on final. 

• Determines your sequence. Make it easier to identify separation problems (too much or not enough). 

• Gave aid in spacing. 
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• In cases where 1 was ahead of the slot I was reminded to review spacing. 

• Useful for separation at the runway threshold. 


10. Rather than the centerline slot marker format, as used in this test, a different centerline format would be...(Please 
indicate any suggestions for improving the display format.) 


not preferable 


CO 

2 

CD 

1 0 


preferable 


Comments 

• No change needed. 

• A single line shown on final with a numeric value display a speed adjustment that would open or close the separation 
to the minimum required separation. This would be displayed once the a/c are on the localizer. 

• I prefer the display. 

11. Having additional aids (such as turn advisories, vector advisories, or speed advisories) to help me deliver aircraft 
into their slot markers would be...(Please indicate your preference, if any.) 


unnecessary 


0 

1 

4 

3 

4 


desirable 


Comments 

• Maybe in addition to the slot markers also the line segments only if the slot marker were going to be something I had 
to work with. 

• If you had to have the slot marker, a graphic turn display would help. 

• Not sure, I would have to try it. It would probably be too much to concentrate on slots, vector advisories, etc (too much 
distraction). 

• Graphic display. 

• Headings could be helpful in some cases but not necessary. Some controllers might prefer that. 

• Graphic markers. 


12. Determining which aircraft were supposed to go into which slot marker was... 


difficult 


0 

0 

1 

2 

9 


easy 


Comments 

• The slot markers explained very easily where aircraft were meant to be. 

• Getting them there was the problem. 

• Most cases seemed natural - I didn’t need the tab list as in vector advisories (Dice). 

• Sometimes the computer sequenced incorrectly. 


13. Relative to the final controller s job, indicate your reaction and feeling to having a computer suggest where an 
aircraft should be positioned on final. 


undesirable 


0 

1 

1 

6 

4 


desirable 


Comments 

None 


14. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was... 


below normal 


0 | 


6 

3 

2 


above normal 


Comments 

• This system takes all my alertness and awareness to perform properly. 
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15. Additional comments on performance or conditions, if any. 


Comments 

• Have some way to change slots. 

• Tended to mentally berate myself if I missed a slot - maybe that would lessen with continued usage. 

• I felt it got away from me because I wasn’t looking ahead to the sequence. 


F.2.2. Format Questionnaires for 210-Knot Pattern-Speed Procedure 

F.2.2.1. Manual Format for 210-Knot Procedure 


1. Given the organization of traffic performed by the feeder and automation in this session, the landing sequence 
was... 


not obvious 


0 

0 0 

4 

8 


readily apparent 


Comments 

• The scheduling system is very accurate and the feeder position was excellent. 

• The test should have a few sequencing decisions. It is unlikely that a controller would work this volume of traffic without 
a tie. 

• During the scenario there were maybe two or three instances where I had to make a judgement call as to which a/c 
would go first. 

• Feeder controller vectored inbounds as necessary to avoid possible conflicts. 


2. In today’s manual TRACON environment there is no automation to aid the feeder in organizing traffic for the final 
controller. Given the aircraft spacing and organization performed by the feeder/automation interaction in this 
session, the effort required to set up the landing sequence as compared to today’s manual environment was... 


greatly increased 


0 

0 

1 

6 

5 


greatly reduced 


Comments 

• Projecting ahead, I feel the automation interaction was better than I could have foreseen. 

• Let’s put it in at ORF. 

• Most controllers do a good job of setting up a workable sequence. 


3. Communication with pseudo pilots was.. .(Please identify any deficiencies.) 


unrealistic 


0 

1 

2 

5 

4 


realistic 


Comments 

• Realistically the communication sequence is not as organized. The pilots did well. 

• Pilots responded accurately and quickly. You have to listen more carefully because of the computer generated different 
type voice. 

4. The interaction and coordination with the feeder controller were...(Please identify any deficiencies.) 


unrealistic 


1 

0 

2 

5 

3 


realistic 


Note: One subject stated there was no interaction during the run and therefore did not respond. 

Comments 

• Although no verbal communication between us existed, this session of the interaction was very good in traffic flow. 
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• Didn’t coordinate with feeder during this scenario. 

• Didn’t really use any this session, but in reality would use a lot. 

• I only made a couple of coordinations, but they were timely and the feeder concurred. 


5. During this data run, the procedure of 210 kts in the pattern followed by a norma! reduction to 170 kts, after the 
turn-to-final, was used, indicate how this procedure compares to what you typically use during IMC, with moderate 
to heavy traffic. (If different please explain.) 


strongly differed 


1 1 5 

4 

1 

1 


closely agreed 


Comments 

• Personally I tend to reduce a/c much sooner, say on base leg or downwind instead of turn to final. 

• I typically will reduce a/c prior to turning final unless 1 am running a long final in excess of 15 miles. I find it’s easier to 
sequence a/c with more compatible speeds. 

• Traffic sometimes is slowed earlier during busy traffic. 

• At Norfolk we often don’t use/need speed reduction as in this situation. There are times we use it, but usually its for 
air carriers succeeding twin or single engine a/c. 

• A speed of 210 to 8.5 DME might be a little fast for some aircraft. You cannot have a 3 mile final for 30 miles using 
this procedure. 

• I tend to use speed reduction very often but towards the end of the problem it felt comfortable. 

• Military, civil mix at ORF and changes greatly how you apply speed adjustments. 

6. Given the traffic situation during this test run, a pattern speed of 210 kts through the turn-to-final was ...(If not 
acceptable, please explain.) 


not acceptable 



acceptable 


Comments 

• It kept the pattern moving and traffic flow was much more expeditious, however, it differs from what I do. 

• For the most part it worked out fine however its final turn to final could be more precise if a/c are slowed to 180 -170 
on base leg. 

• The speed of 210 kts is very acceptable. They are easy to use when computing distances traveled. 

7. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was.. .(If not normal, piease indicate reason.) 


below normal 


0 


above normal 


Comments 

• I felt like I normally felt in a manual environment, no real change. 

• I felt fatigued the second session. 

• The fact that I was being watched made me be more aware and pay more attention. 

• It is after all, a test scenario. If I was working live traffic, I would have other means at my disposal. 

8. Additional comments on performance or conditions, if any. 

Comments 

• I felt my overall ability to control this particular simulated pattern was greatly enhanced by the automation aids I had 
used in the previous runs. 

• In my normal work environment speed control is not used very often. This was somewhat of a learning experience to 
use speed control with every aircraft. 

• Speeds seem to drop too fast inside of the marker. 


Q-z. 
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F.2.2.2. Graphic Marker Format for 210-Knot Procedure 


1. Adapting to the use of the graphic turn advisor with the 210 kts procedure was... 


difficult 


0 

1 

0 

5 

6 


easy 


Comments 

• By far the best combination automation I’ve worked with yet. 

• At times it seemed more confusing than the 170 kt graphic turn scenario - maybe it was me. The graphics hopped 
around a few times. 

• It was very easy adapting to the graphic turn advisor. Little or no concentration needed. 


2. The turn positions indicated by the graphic turn advisor, as compared to the positions you would have turned 
aircraft for spacing on final... 


strongly differed 


0 

1 

4 

4 


3 


closely agreed 


Comments 

• They were very accurate turns by the advisor and I believe mine would have overall continued. 

• I would normally run closer downwind, therefore, I probably would have adjusted the downwind heading. 

• There were a number of times I’d have turned base later, so as not to get into an overshoot of the localizer for proper 
sequence. 


3. Indicate how the graphic turn advisories (210 kts procedure) affected your workload , as a final controller, as 
compared to manual operation (210 kts procedure) with no computer aid. 


increased 


0 

CO 

1 

5 

CO 


reduced 


Comments 

• I had much more time to scan more airspace and better monitor final. 

• Workload was reduced a lot. At one point I was lulled to sleep. 


4. Indicate how the graphic turn advisories (210 kts procedure) affected your spacing on final approach as compared 
to manual operation (210 kts procedure) with no computer aids. 


less precise 


0 

0 

0 

9 

3 


more precise 


Comments 

• Once again almost too accurate in getting lulled into a false sense of security. However, with the turn and speed 
advisories so accurate I was able to better watch final spacing close in at my own pace and technique. 

• A couple of times I noticed spacing near the threshold slightly less than 2 1/2-1 don’t know if it was because of me or 
the graphic turn advisor. 


5. In terms of display clutter, the addition of the graphic turn marker symbol to the display generated... 


excessive clutter 


0 

4 

5 

1 

2 


no perceptible clutter 


Comments 

• Even in this environment there was a lot of lines and markers but that is a small distraction to the overall usefulness 
of the advisor. 

• I fell behind the computers suggested turns at that time when I was catching up. The graphic turn marker was a minor 
distraction. 
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• Not really a problem. 

• Again, with a cluttered map, it could be a problem. 


6. Indicate how your focus on the aircraft/graphic-turn-marker position relationship affected your attention to aircraft- 
to-aircraft spacing. 


strongly reduced 


1 



4 3 


not affected 


Comments 

• Separation on final was so constant my attention was very evenly distributed. 

• Minimal reduction in a/c awareness. 

• I paid a lot of attention to the X and the graphics - especially if they moved a lot. 

• I did not watch the spacing as closely as I would have with the manual operation. 


7. Indicate how your focus on the aircraft/graphic-turn-marker position relationship affected your attention to the 
overall traffic picture. 


strongly reduced 



not affected 


Comments 

• I was better able to watch all aspects of my airspace because of the advisories efficiency. 

• Attention was drawn toward the turn monitor to see if aircraft was following the turn. 


• Slightly reduced. 

• I tried to force myself to see the overall picture. 

8. Relative to the final controller’s job, indicate your reaction and feeling to having a computer suggest where aircraft 
should be turned for spacing. 


undesirable 


0 

1 

2 

6 

3 


desirable 


Comments 

• If I could have this system at Norfolk right now! 

• With additional practice, could be helpful. Difficult to tell with just this session. 

• It was nice to have something assist you working the final. 


9. The speed reduction points indicated by the graphic speed-reduction advisor, as compared to the positions you 
would have slowed aircraft for spacing on final... 


strongly differed 


0 

1 

0 

8 3 


closely agreed 


Comments 

• I saw no drawbacks and agreed totally. 

• I tended to reduce the aircraft about 1 mile sooner than the graphic. 

• The advisor was later than what I would have done. 


10. indicate how the graphic speed-reduction advisories (210 kts procedure) affected your workload , as a final 
controller, as compared to manual operation (210 kts procedure) with no computer aid. 


increased 


0 

C\J 

6 

3 


reduced 


Comments 

• My job with this system is nothing more than a monitor, therefore, I could spend much more time on other responsibilities 
in my airspace. 

• Made the job easier. Less pre-planning required. 
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11. Indicate how the graphic speed-reduction advisories (210 kts procedure) affected your spacing on final approach 
as compared to manual operation (210 kts procedure) with no computer aids. 


less precise 


0 

0 

2 

7 

3 


more precise 


Comments 

• As precise as I’ve seen - simple, straightforward advisory. 

• Not sure. 

• I would have slowed the aircraft a little earlier than the aid. 


12. In terms of display clutter, the addition of the graphic speed-reduction marker symbol to the display generated... 


excessive clutter 


no perceptible clutter 


Comments 

• More than I’m used to but a minor drawback. 


13. Relative to the position were aircraft speeds were to be reduced to 170 kts, the graphic speed advisories appeared 
ahead of the aircraft at a time that was... 


too early 


0 

0 

11 

1 

0 


too late 


Comments 

• Very satisfactory. 

• The speed reduction for a/c behind a heavy jet seemed a little early. 

• Several times when the advisor appeared it was already under the aircraft. 

• In a few or more cases there didn’t seem to be a “heads up". 

• The graphic speed advisory was generally on early enough to see it, sometimes a little too early. 


14. Indicate how your focus on the aircraft/graphic-speed reduction-marker position relationship affected your 
attention to aircraft-to-aircraft spacing. 


strongly reduced 


0 

1 

1 

8 

2 


not affected 


Comments 

• I was able to monitor spacing much more accurately with this system than before. 

• My scan was less effected by speed than by the dice countdown. 


15. Indicate how your focus on the aircraft/graphic-speed reduction-marker position relationship affected your 
attention to the overall traffic picture. 


strongly reduced 


1 

0 

1 

8 

2 


not affected 


Comments 

• Because of the system efficiency I was free to attend to more airspace - more quality overall. 


16. Relative to the final controller s job, indicate your reaction and feeling to having a computer suggest where aircraft 
should be reduced (from 210 kts) for spacing. 


undesirable 


0 

1 

0 

6 

5 


desirable 


Comments 


• If it reduces workload like this system does, yes. 

# If the X was a different color or blinked until speed reduction was accomplished maybe it would be good. 
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17. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was. ..(If not normal, please indicate reason.) 


below normal 


0 5 


0 


above normal 


Comments 

• This system relaxes you and helps you maintain an even, positive mental posture. 

• With the computer making all the decisions for me, my awareness is not as focused. 

18. Additional comments on performance or conditions, if any. 

Comments 

• I feel the markers would be less distracting and cause less clutter if they were disabled earlier, i.e., as soon as aircraft 
started their turn. 

• Overall adds clutter to display, not unmanageable but more than 1 like. 

• This was the easiest and most advantageous system for me. 


F.2.2.3. DICE Countdown Format for 210-Knot Procedure 


1. Adapting to the use of the DICE turn advisor with the 210 kts procedure was... 


difficult 


0 

1 

1 

3 


easy 


Comments 

• Very adaptable and accurate in spacing, at 210 kts it was a snap to use. 

• I learned to trust the advisor after the first run and learned when the overshoot advisory could be compensated for with 
speed reduction. 

2. The times-to-turn Indicated by the DICE turn advisor, as compared to the times you would have turned aircraft for 
spacing on final... 


strongly differed 


closely agreed 


Comments 

• Only once or twice did I trust my instincts instead of the DICE and that was during overshoots. 

• Didn’t know. 


3. Indicate how the DICE turn advisories (210 kts procedure) affected vour workload , as a final controller, as compared 
to manual operation (210 kts procedure) with no computer aide. 


increased 


reduced 


Comments 

• Here in simulation my workload was very light in that all I was concerned with was base or downwind turns. 

• I liked the speed reduction prompt. It required less concentration. 

• Slightly more than average because I had to dwell on whose DICE instruction took priority. I also had to consciously 
remember (remind myself) to check spacing, overall picture etc. 

4. Indicate how the DICE turn advisories (210 kts procedure) affected your spacing on final approach as compared 
to manual operation (210 kts procedure) with no computer aids. 


less precise 


0 

2 

6 

4 


more precise 
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Comments 

• The spacing just about always was exact. The only drawback would be ignoring final spacing on my part because of 
the efficiency of the DICE. 

5. In terms of display clutter, the DICE-turn-advisor countdown value and suggested-heading value which are added 
to the data block, generated... 


excessive clutter 


0 

1 

0 

4 

7 


no perceptible clutter 


Comments 

• Not in this environment, with a real sector it could be increasingly cluttered. 

6. Indicate how your focus on the DICE-turn-advisor countdown value affected your attention to aircraft-to-aircraft 
spacing. 


strongly reduced 


1 

4 

2 

CO 

2 


not affected 


Comments 

• The spacing was always taken care of by DICE except if I was to have an overshoot then I would reduce the speeds 
of aircraft earlier than suggested. 

• Once again if I reduced the a/c when I was prompted to do so, I felt my sequence would hold to the airport and did 
not recheck as often as I normally would. 

• Concentrating on the countdown rates decreased my focus on aircraft inside the marker. 

7. Indicate how your focus on the DICE-turn-advisor countdown value affected your attention to the overall traffic 
picture. 


strongly reduced 


0 

4 

3 

4 

1 


not affected 


Comments 

• I did have to pay extreme attention to the speed indicators (more than I would normally) to ensure proper spacing. 

• After learning the timing of each countdown DICE was a big help. 

• Didn’t watch the final inside the marker as closely as in manual mode. 

• At first I was trying to pay too much attention to the turn advisor on downwind to base. 

8. Relative to the final controller s job, indicate your reaction and feeling to having a computer suggest when a 
aircraft should be turned for spacing. 


undesirable 



desirable 


Comments 

• It is interesting to see how the computer agrees or disagrees with what headings I would use. 

9. The speed reduction times indicated by the DICE speed-reduction advisor, as compared to the times you would 
have reduced aircraft speed for spacing on final... 


strongly differed 



closely agreed 


Comments 

• The DICE was probably more accurate than my own spacing. 

• On 2 or 3 occasions I would have reduced aircraft on base. 
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10. Indicate how the DICE speed-reduction advisories (210 kts procedure) affected your workload , as a final controller, 
as compared to manual operation (210 kts procedure) with no computer aide. 


increased 


0 

1 

1 

8 

2 


reduced 


Comments 

• I was always confident in proper spacing and speed due to DICE, therefore, I had a decrease in workload. 

• Didn’t really reduce it, because instead of looking at spacing and deciding when to reduce speed - I looked at DICE 
advisories and had to decide which was higher priority - then, if it was late reducing - I worried that spacing might be 
off. 

11. Indicate how the DICE speed-reduction advisories (210 kts procedure) affected your spacing on final approach as 
compared to manual operation (210 kts procedure) with no computer aid. 


less precise 


0 

0 

CO 

CO 

6 


more precise 


Comments 

_• As precise as I have ever seen in air traffic control. 

• Under a controlled environment, I would not be able to compete. 

• HA! - HA! 

• How about changing the heading above to “too precise”. I'm not that good. 

12. In terms of display clutter, the DICE-speed-reduction countdown value and suggested-speed value which are 
added to the data block, generated... 


r~o 

excessive clutter 1 



no perceptible clutter 


Comments 

• Not in this simulation, in a real radar sector maybe. 

• Flashing caused a bit more clutter - but necessary. 

13. Relative to the time when aircraft speeds were to be reduced to 170 kts, the DICE speed-reduction advisories 
appeared in the data block at a time that was... 


too early 


o 1 

2 

9 

1 

0 


too late 


Comments 

• The advisories appeared to give me ample time unless I had altered the scenario. 

• In some cases the speed reduction advisory flashed prematurely (when a/c on an already assigned heading was given 
an approach clearance). 

14. Indicate how your focus on the DICE-speed-reduction countdown value affected your attention to aircraft-to-aircraft 
spacing. 


strongly reduced 


_U 

5 

2 

4 

0 


not affected 


Comments 

• I was most likely to be too confident that the aircraft were separated. 


15. Indicate how your focus on the DICE-speed-reduction countdown value affected your attention to the overall traffic 
picture. 


strongly reduced 


not affected 
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Comments 


None 


16. Relative to the final controller’s job, indicate your reaction and feeling to having a computer suggest when aircraft 
should be reduced (from 210 kts) for spacing. 


undesirable 


0 

0 

1 

6 

5 


desirable 


Comments 

• Based on personal experience it would be hard in a normal sector to spend that much time monitoring the data blocks 
effectively. 

17. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was.. .(If not normal, please indicate reason.) 


below normal 


0 

0 

7 4 

1 


above normal 


Comments 

• I was more alert to when to slow aircraft and turn them due to the printout nature of the DICE. 

• Attention was directed to the a/c that was next on final and turning base, while a/c on final were not really being 
scanned. 

18. Additional comments on performance or conditions, if any. 

Comments 

• Although concentrating on the countdown time took away from my scan inside the marker, I found that when I did scan 
the area, traffic was spaced well in most cases. 

• I do not normally give 90 degree turns to final. 

• Good aid. 

• The DICE simulation was much easier to work than the manual simulation. It did reduce my attention of the spacing 
of the traffic because I was watching the countdown. This simulation was fine in a sterile situation. 

• With the addition of a tower controller monitoring the final, this program could greatly reduce separation in the pattern 
on final approach. It’s a good training aid also. 


F.2.2.4. Centerline Slot Marker Format for 210-Knot Procedure 


1. Adapting to the use of the Centerline Slot Marker with the 210 kts procedure was.. 

0 I 5 I 4 P3 0 
difficult easy 


Comments 

• Still a very skill-oriented procedure, the final turn-ons were easier, however there was still great effort required to turn 
accurate base leg turns. 

• I see where the slot marker could become beneficial, however adjusting to it was frustrating. 

• I felt that I got better with practice and could do this on an everyday basis. 

2. Indicate how the Centerline Slot Marker (210 kts procedure) affected your workload, as a final controller, as 
compared to manual operation (210 kts procedure) with no computer aids. 


0 


5 


6 


0 


increased 


reduced 







Comments 


• Increase in work due to the mental calculations involved with turns and closure rates. 

• The slot marker initially is more of a hindrance than a help. 

• The moving slot markers seem to distract me from concentrating on traffic inside the marker, however, it gave me a 
fairly consistent final and a good point to apply speed control. 

• Required a great deal of concentration on slots and when to reduce the aircraft. I looked less at spacing between a !c 
and more on whether they hit the slot. 

• Using the slot marker was harder than using the DICE, but easier than the manual system. 


3. In the normal case (when slot markers were not noticeably shifted to adjust for spacing errors) vectoring aircraft 
into their slot markers with the 210 kts procedure was... 


difficult 


easy 


Comments 

• Vectoring to final spacing was the easiest aspect of the system. 

• On one occasion I can remember the slot marker shift helping me out. The remainder of the time I felt behind. 


4. Indicate how often an extra effort was made to precisely center the aircraft in its slot marker. 


never 


1 1 2 [ 6 | 3 


always 


Comments 

• Speed reductions were in order, or more precise turns were needed for the centering to occur. 

5. In terms of display clutter, the slot markers displayed on the extended runway centerline generated... 


excessive clutter 


0 

2 

6 

1 

3 


no perceptible clutter 


Comments 

• For this environment not any distraction. 

• I do remember having to wait to read a call sign I had forgotten when I missed the slot marker in front. 

• Not much of a problem on centerline. 


6. Indicate how your focus on the aircraft/slot-marker-position relationship affected your attention to alrcraft-to- 
alrcraft spacing. 


strongly reduced 


1 

6 

1 3 

1 


not affected 


Comments 

• By using the system properly aircraft to aircraft spacing was assured more often. 

• I noticed that l was less concerned with a/c inside the marker. 


• I counted on the fact that if they were in the slot, they were spaced well. Only if I missed the slot (ahead or behind) 
did I scrutinize the spacing. 

• As in the Dice system, I did not watch the traffic as close as I would have with the manual system. 

7. Indicate how your focus on the aircraft/slot-marker-position relationship affected your attention to the overall 
traffic picture. 


strongly reduced 


1 

5 

1 

5 

0 


not affected 


Comments 

• This type system really requires great attention to detail and focus. 

• This is new, old controllers say they used to turn the ARTS off when they got busy. When it initially came out. 
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8. In spacing/sequencing aircraft on final, to what degree do you feel the slot markers provide useful information. (If 
useful, how?) 


not useful 


0 

C\J 

0 

00 

2 


useful 


Comments 

• Spacing suggested was extremely accurate. 

• I know when I miss a hole, now everybody else does. 

• It gave a reference point to apply space adjustment. 

• Gave indication of spacing. 

• They give guidance as to correct spacing - especially good with heavy a/c. 

• Once you get used to it you can go on to other duties after the a/c is positioned in the marker. 

• Projection/looking ahead. 

9. Having additional aids (such as turn advisories, vector advisories, or speed advisories) to help me deliver aircraft 
Into their slot markers wouid be. ..(Please indicate preference, if any.) 


unnecessary 


2 

1 

2 3 

4 


desirable 


Comments 

• As I mentioned before possibly coupled with line segments. 

• A turn advisory could initially help. 

• Speed reductions. 

• Heading and speed advisories would be useful. 

• Too much to think about. 

• Vectors to hit slot markers would definitely be an aid. 

• Speed control would be very advantageous and maybe a mark for a normal base leg to final with mileage markers. 


10. Determining which aircraft were supposed to go into which slot marker was... 


difficult 


0 

0 

1 

0 

11 


easy 


Comments 

• Self-explanatory tags in the circles. No problem! 

• It required a slightly greater effort to assure proper a/c in proper slot. 


11. Relative to the final controller s job, indicate your reaction and feeling to having a computer suggest where an 
aircraft should be positioned on final. 


undesirable 


desirable 


Comments 

• With the sequence so obvious we agreed on all calls. 

• Hard to say - would require more practice. 

• It is desirable in a sterile situation. 


12. Relative to my normal state of alertness, awareness, and responsiveness, while controlling traffic, I feel my state 
in this session was... 


1 


5 


3 


below normal 


above normal 








Comments 


• With this system I am very alert and tend to even lean toward aggressiveness to achieve proper spacing and system 
success. 

13. Additional comments on performance or conditions, if any. 

Comments 

• The 210 format is preferable to the 170 because it allows some flexibility. 

• Problem became easier towards the end as I was more comfortable with how to control the traffic to hit the slots. 

F.3. Task Load Index — Individual Subject Controller Results 

F.3.1. TLX Description 

The primary questionnaire used to assess workload was the Task Load Index (TLX). The TLX is a 
multidimensional rating procedure that provides an overall workload score based on weighted averages. 

At the completion of each run, the subject rates the experience during the run for each of six workload 
contributing factors on a scale consisting of 20 increments. The factors are mental demand (MD), physical 
demand (PD), temporal demand (TP), own performance satisfaction (OP), your effort required (ER), and 
frustration you experienced (FE). The rating sheet used for the evaluation is as follows: 

Instruction : Circle the vertical graduation mark on ech scale that indicates your relative rating of the following six workload- 
contributing factors, as experienced in the task just performed. 

FACTORS ARISING FROM TASK ITSELF 

MENTAL DEMAND 

I I I I I I I I I I I I I I I I L I _L_ 

Low High 

PHYSICAL DEMAND 


Low High 

TEMPORAL DEMAND 


Low High 

FACTORS ARISING FROM YOUR INTERACTION WITH TASK 

OWN PERFORMANCE SATISFACTION How success f u | <j 0 y 0U think you were in accomplishing the 

,|jl . | | | goals of the task set by the experimenter (or yourself)? How 

1 1 i I I 1 I 1 I 1 1 1 1 1 I 1 1 I 1 1 satisfied were you with your performance in accomplishing 

Good Poor these goals? 


How much mental and perceptual activity was required 
(e.g. thinking, deciding, calculating, remembering, looking, 
searching, etc.)? Was the task easy or demanding, simple 
or complex, exacting or forgiving? 

How much physical activity was required (e.g. pushing, 
pulling, turning, activating, etc.)? Was the task easy or 
demanding, slow or brisk, slack or strenuous, restful or 
laborious? 


How much time pressure did you feel due to the rate or pace 
at which the tasks or task elements occured? Was the pace 
slow and leisurely or rapid and frantic? 
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YOUR EFFORT REQUIRED 

Flow hard did you have to work (mentally or physically) to 

| | | | | 1 | I ] Mi l | | | j ]__ accomplish your level of performance? 

Low High 


FRUSTRATION YOU EXPERIENCED 



Low High 


How insecure, discouraged, irritated, stressed and annoyed 
verses secure, gratified, content, relaxed and complacent 
did you feel during the task? 


At the completion of all data runs, the subject is tasked with making “pairwise” comparisons of the six 
different workload contributing factors by selecting the preference of a factor pair in each of 15 comparisons. 
The result of the pairwise comparisons is a “weight” representing the relative degree to which the subject 
feels that each factor has contributed to the workload of the task. The matrix used in making the pairwise 
comparison is as follows: 


Pairwise Comparison of Sou rce-of- Workload Factors 
Factor Pairs 


Effort 

Own 

Physical 

Frustration 

Mental 

Physical 

required 

performance 

demand 

experience 

demand 

demand 

Temporal 

Effort 

Physical 

Temporal 

Frustration 

Mental 

demand 

required 

demand 

demand 

experience 

demand 

Own 

Frustration 

Temporal 

Mental 

Own 

Mental 

performance 

experience 

demand 

demand 

performance 

demand 

Physical 

Own 

Frustration 

Effort 

Mental 

Effort 

demand 

performance 

experience 

required 

demand 

required 

Temporal 

Frustration 

Own 

Temporal 

Effort 

Physical 

demand 

experience 

performance 

demand 

required 

demand 


F.3.2. Individual Subject Controller Results 

The graphed data for each subject/data run is provided in figure FI. The width of the bars represent the 
weight as determined by the pairwise comparison; the maximum width for any of the factors is five increments. 
The height of each bar is the result of the ratings performed at the end of each run. Note that on all graphs 
only five of the six workload factors are represented; this is because all of the subjects felt that physical demand 
(PD) was less of a workload contributing factor than each of the other five factors in their pairwise comparisons. 
For greater detail on the TLX refer to reference 21. 

F.4. Rating and Ordering Test Formats — Questionnaire Only 

Subjects rated and ordered the test formats on this questionnaire with respect to three specific criteria: 
workload or effort required, ease of adapting to or learning to use the format, and the degree of help or benefit in 
spacing aircraft on final. Only the instructions and form are given. The responses are presented in section 5.5.3. 

Based on your experience and judgment as a controller, please rate and order the formats that you were tested with on the 
scales below, with respect to each of the following three criteria: 

1. Workload or effort required to use the format. 
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2. Ease of adapting to or learning to use the format. 

3. Amount of help or benefit in spacing traffic on final. 

With respect to the criteria being evaluated, place the format(s) symbol at each end of the scale which you feel best fit(s) the 
endpoint description. Arrange the remaining formats along the scale to reflect your evaluation of the formats with respect to 
your endpoint choices. Note that you can evaluate two or more formats equally, either at the endpoints or any position on the 
scale, by placing their symbols in the same scale location. This is not an absolute rating relative to some ideal format, what 
we are after is your comparative rating of the tested formats with respect to each other. 

Example for discussion with coordinator. 

For instance, let us say 8 formats have been tested: A, B, C, D, E, F, G, and H. Assume they are being rated and ordered on 
the workload criteria and format C has the least workload, while format E requires the most workload. If you feel the formats 
are fairly evenly spaced with respect to workload, the scale would look something like: 


most workload 







B 





E 



H 


D 


F 

A 

C 


least workload 


On the other hand, if you felt one format stood out as having a much higher workload relative to all the other formats, the 
scale would look something like: 


most workload 








G 

B 


A 

E 






H 

D 

F 

C 


least workload 


The symbols to be used on the following scales are: 

Ml - Manual or no automation aid (170 kts final turn) 

M2 - Manual or no automation aid (210 kts final turn) 

G1 - Graphic position-to-turn advisor (170 kts final turn) 

G2 - Graphic with both position-to-turn and position-to-speed reduction advisories (210 kts final turn) 

51 - Centerline slot marker (170 kts final turn) 

52 - Centerline slot marker (210 kts final turn) 

D1 - DICE time-to-turn advisor (170 kts final turn) 

D2 - DICE with both time-to-turn and time-to-speed reduction advisories (210 kts final turn) 

1. Workload or effort required to use the format. 


most workload " least workload 


2. Ease of adapting to or learning to use the format. 


hardest to adapt easiest to adapt 


3. The degree of help or benefit in spacing aircraft on final. 


least helpful most helpful 


F.5. Comparison of Format Questionnaires for 170- and 210-Knot Pattern-Speed 
Procedures — Subject Evaluations and Comments 


The questionnaire and results of the comparison between the 170- and the 210-knot pattern-speed 
procedures, for each format, are presented in this section. Each comparison includes the total number of 
responses for each of the five positions along the scale, as well as all subject comments. 
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1. For the manual cases with no automation aids, evaluate the level of difficulty associated with spacing aircraft on 
final using a speed of 210 kts and reducing after the turn-to-final as compared to the constant 170 kts before the 
turn-to-final. 


much harder 


0 

2 

1 

6 

3 


much easier 


Comments 

• Same speeds manually has no guesswork, at 210 kts there is a bit of an increase. 

• Speed can be used to cover up a late turn-on. 

• When you tie someone down to one speed, you take a valuable tool from a controller. 

• Due only to the fact this is a procedure not practiced in the field. It would be otherwise to controllers who constantly 
do this. 


2. For the graphic advisor cases, evaluate the level of difficulty associated with spacing aircraft on final using a 
speed of 210 kts for the turn-to-final together with both the position-to-turn and the position-to-reduce speed 
advisories as compared to a speed of 170 kts for the turn-to-final with only the position-to-turn advisory 


much harder 


0 

CO 

2 

4 3 


much easier 


Comments 

• Graphic advisor with speed reduction at 210 kts keeps things moving expeditiously, safely, and more accurately than 
any of the other systems. 


3. For the centerline slot marker cases , evaluate the level of difficulty associated with vectoring aircraft to their 
slots using a speed of 210 kts and reducing after the turn-to-final as compared to a constant 170 kts before the 
turn-to-final. 


much harder 


much easier 


Comments 

• Difficult system either way, slightly easier with 2f0 kt format. 


4. For the DICE countdown advisor cases , evaluate the level of difficulty associated with spacing aircraft on final 
using a speed of 210 kts for the turn-to-final together with both the time-to-turn and the time-to-reduce speed 
advisories as compared to a speed of 170 kts for the turn-to-final with only the time-to-turn advisory 


much harder 


1 

2 

3 

4 

2 


much easier 


Comments 

• This system works well either way, a slight edge to reduce time but overall a steady system which could work well in 
select facilities but not all. 

• In both the graphic and dice, the simulation ran easier at 170 kts because the closure rates on the turn indicators was 
slower, however, if you were ever late it was easier to recover in the 210 kt scenario. 

• Of all the formats the DICE is much superior to the others. 


F.6. Final Debriefing Questionnaire — Subject Evaluations and Comments 


The Final Debriefing Questionnaire contains general questions about the simulation and training. The 
questionnaire and the total number of responses for each of the five positions along the graphic scale are 
included, as well as all controller comments. 
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1. Overall, the simulation was... 


unrealistic 


realistic 


Comments 

• Normally a controller will work more than depicted but for this purpose of evaluation it was fine. 

• Very good as far as simulation-would be more realistic with flight progress strips but did not effect performance. 

• Missed various types of a/c. 

• Very sterile operation. You need to possibly put some distractions in the problem. 

2. The suitability of the simulated radar display to perform the required control task was...(Please identify any 
deficiencies.) 


seriously deficient 


0 _ 


0 

1 

3 

8 


adequate 


Comments 

• I normally don’t work with computer generated scopes and it was a pleasure having clarity of the scope. 

• Controllers are used to having the capability of adjusting their scope to personal specs. - except for that it was OK. 

• You could keep track of where the a/c was, but there was no beacon code slash or primary return. 

• More mistakes (pilots) in real life, but ail in all a good simulation. 


3. Indicate how the physical environment, during the test, influenced your test performance. 


degraded 


0 

1 

8 

1 

2 


improved 


Comments 

• No real distraction and the lighting was the same, no influence felt. 

• Conducive to concentrating more quiet than ORF IFR radar (room). 

• Ocular LIGHT a bit distracting. 


4. The voice communications with the pseudo pilots were.. .(Please identify any deficiencies.) 


unrealistic 


0 

4 

3 1 

4 


realistic 


Comments 

• Because of the basic nature of the simulation I had no problems with them, but they could have been more attentive 
to respond. 

• To make it more realistic once a pilot starts talking they should not stop until their transmission is complete. 

• When instructions were issued to wrong aircraft. ..pilot didn't question instructions. 

• Timing too quick not many request to repeat. 

• Voice quality of the pilots sounded like martians. The pilots made very few errors and acknowledged all transmissions 
quickly. 


5. The simulated aircraft flight paths and maneuvers were...(Please identify any deficiencies.) 


unrealistic 


realistic 


Comments 

• Turn to final ratio appeared slower at first, I guess due to the elevation of the terrain. 

• A bit too ideal, but fine to work with. 
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• A DC-8 would have a tough time getting down from the SE and NE fixes. Some a/c were slowing to 1 10 kts 2 - 3 miles 
out. High performance a/c don’t fly that slow that far out. 

• Speeds inside marker seemed to drop off fast. 


6. The traffic density in the simulation as compared to that expected In a high density, single runway terminal 
operation in IMC was... 


not representative 




representative 


Comments 

• I don’t feel qualified to say, however, it was moderate to busy traffic in my opinion. 

• Normally there would be a few lower performance a/c in the sequence. 

• Not used to this type of traffic. 

• With the exception that all a/c in the simulation were a/c without a mix of civilian smaller aircraft. 

• Did not work in a facility like Denver. 

• You could not run that type of traffic on a single runway with any departures to go. 

7. The interaction and coordination with the feeder controller was.. .(Please identify any deficiencies.) 


unrealistic 


realistic 


Comments 

• The only thing lacking would be maybe an interphone system. 

• There was very little coordination. 

• Didn’t use as much as would be expected. 


8. The initial briefing and training on Denver airspace/procedures and the simulation interface necessary to do a 
representative job of controlling traffic was.. .(Please identify any deficiencies.) 


seriously deficient 


0 0 1 


1 1 10 


adequate 


Comments 

• No problems. 

• I would like to have the ILS 26L approach plate next to me and look at the profile descent (STAR). 


9. The briefing I received describing each format prior to the practice runs was...(Please indicate any deficiencies.) 


seriously deficient 


0 

0 

0 

0 


12 


adequate 


Comments 

• Very informative and to the point. 


10. The training 1 received during the format practice run before each test session was. ..(Please identify any 
deficiencies.) 


seriously deficient 


12 


adequate 


Comments 

• I was comfortable after every practice run. 

11. The test sessions were conducted in a controlled, serious and professional manner. 


disagree 


12 


agree 


99 






Comments 


• Very professional. 

• Very professional group; it was my pleasure to be S-2. 

• Everyone was amiable and professional. 

• Too starchy 

12. In the automated aid formats tested, the arrival aircraft’s tabular sequence list presented on the right of the display 
was...(lf useful, how was it used?) 


not useful 


7 

2 

0 

1 2 


useful 


Comments 

• The whole time I was involved in the simulation I never used it. 

• I never used the tab list or felt a need to use it. 

• I occasionally used it to confirm a/c sequence. 

• Only necessary in maybe one - didn’t use it much if at all in others. 

• Annoying, distracting. I disagreed with the computer sequence at the time and it would mess up the program if I did 
not adhere to the computer program. 

• In almost all cases the approach sequence was obvious. 

• I relied on it for the sequence at all times and adjusted the pattern to comply with it. 

13. List any factors not addressed above that either positively or negatively affected your performance. 

• A minor detail was some background light reflecting off the radar display. 

• Enjoyed!! 


F.7. Researcher Topic Guide for Final Verbal Debriefing 

The verbal debriefing was the final opportunity to extract the subjects thoughts and opinions in several areas 
including the formats, the simulation, and test procedures. The subject guide used for the verbal debriefing is 
as follows: 

Do you have any suggestions for improvements or other comments relative to the following; 

1) Centerline slot marker 

2) Graphic turn advisor 

3) Graphic speed reduction advisor 

4) DICE time-to-turn advisor 

5) DICE time-to-speed reduction advisor 

6) Test procedures or test facility 


Was any information missing that you think would be useful? 

Did any of the information cause clutter or distractions? How about fixation? How would you suggest solving this problem 
(delete information, move information somewhere else, make information appear on request only)? 
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What is your reaction to the DICE or Graphic early warning that the automation anticipates a localizer overshoot, to maintain 
separation, if the current heading and speed is maintained? 

Do you feel that you had enough time to develop proficiency in using the system? 

Do you feel that if you were using the system daily, you might evolve a different way of using it than you did in the short period 
of the testing? 

Did the system change the mental tasks involved in controlling traffic? How? 

Do you feel that an automated advisory system will create any special problems with controller training or proficiency? If so, 
explain. 

Do you feel that an automated aid will be beneficial to: 

1) Reducing workload? 

2) Spacing precision? 

Does the reduction from 210 to 170 kts after the turn-to-final approximate the final approach procedure used at your facility? 
Explain. 

Given the traffic rate during the test runs, was a 210 kts pattern speed with speed reduction to 170 after the final turn acceptable 
to manage the final traffic? 

Do you have any thoughts, opinions, or suggestions that were not previously covered? 
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Figure FI. TLX workload ratings for all subject controllers. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure FI. Continued. 
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Figure Fl. Concluded. 
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Table 1. Approximate Schedule for Subject Controller 
[Asterisk and bold run number denote data run (1 hr and 10 min)] 


Tuesday — 
7:45 
8:30 
9:15 
10:15 
*10:35 
11:45 
12:00 
12:50 
1:50 
* 2:10 
3:20 


170 (or 210) knots 

Conference room briefing (airspace, procedures, etc.) 

Equipment introduction, discussion, and oculomctcr calibration 
Manual practice run (run 1) 

Break 

Manual baseline data run (run 2) 

Session questionnaire 
Lunch 

Graphic, centerline slot marker, or DICE format explanation, familiarization run (run 3) 
Break 

Graphic, centerline slot marker, or DICE format data run (run 4) 

Session questionnaire 


Wednesday 170 (or 210) knots 

8:15 Centerline slot marker, DICE, or graphic format explanation, familiarization run (run 5) 
9:15 Break 

*9:30 Centerline slot marker, DICE, or graphic format data run (run 6) 

10:40 Session questionnaire 

11:00 Lunch 

12:00 DICE, graphic, or centerline slot marker format explanation, familiarization run (run 7) 
*1:00 Break 

1:15 DICE, graphic, or centerline slot marker format data run (run 8) 

*2:25 Session questionnaire 

2:45 170- or 210-knot questionnaire 


Thursday — 210 (or 170) knots 


8:15 

Manual practice run (run 9) 


9:15 

Break 


*9:30 

Manual baseline data run (run 

10) 

10:40 

Session questionnaire 

11:00 

Lunch 


12:00 

Graphic, centerline slot marker, 

or DICE 

1:00 

Break 


*1:15 

Graphic, centerline slot marker, 

or DICE 

2:25 

Session questionnaire 



format explanation, familiarization run (run 11) 
format data run (run 12) 


Friday — 210 (or 170) knots 

8:15 Centerline slot marker, DICE, or graphic format explanation, familiarization run (run 13) 
9:15 Break 

*9:30 Centerline slot marker, DICE, or graphic format data run (run 14) 

10:40 Session questionnaire 

11:00 Lunch 

12:00 DICE, graphic, or centerline slot marker format explanation, familiarization run (run 15) 
1:00 Break 

*1:15 DICE, graphic, or centerline slot marker format data run (run 16) 

2:25 Session questionnaire 

2:45 210- or 170-knot questionnaire 

3:15 Final questionnaire, discussion, and debriefing 

5:00 Conclusion 



Table 2. FASA Format Traffic Samples 


(a) 170-knot pattern-speed procedure 


Run type 

Record 0 

Sample 

Start time, sec 

Stop time, sec 

Duration 

Practice 

17-8 

423EHR 

5416 

9016 

1 hr 

Data 

17-2 

421GHR 

3 632 

7832 

1 hr 10 min 

Data 

17-3 

422FHR 

3 632 

7832 

1 hr 10 min 

Data 

17-4 

421GHR 

7284 

11484 

1 hr 10 min 

Data 

17-5 

422FHR 

7 236 

11436 

1 hr 10 min 

Data 

17-6 

421GHR 

10 868 

15 068 

1 hr 10 min 

Data 

17-7 

422FHR 

10 820 

15 020 

1 hr 10 inin 


a In record number, 17 indicates 170 knots. 


(b) 210-knot pattern-speed procedure 


Run type 

Record 0 

Sample 

Start time, sec 

Stop time, sec 

Duration 

Practice 

21-8 

3E 

5 424 

9024 

1 hr 

Data 

21-2 

1G 

3 620 

7820 

1 hr 10 min 

Data 

21-3 

2F 

3 644 

7844 

1 hr 10 min 

Data 

21-4 

1G 

7224 

11424 

1 hr 10 min 

Data 

21-5 

2F 

7 281 

11484 

1 hr 10 inin 

Data 

21-6 

1G 

10836 

15 036 

1 hr 10 min 

Data 

21-7 

2F 

10816 

15 016 

1 hr 10 min 


°In record number, 21 indicates 210 knots. 


Table 3. FASA Format Study Test Sequence 

Subject numbers are in original test order and do not correspond to subject numbers used to 
indicate subject performance; even run numbers denote experimental data runs, whereas 
odd numbers (not shown) denote controller practice run; in record numbers, 17 and 21 
denote 170- and 210-knot pattern-speed procedures, respectively. 




170 (or 210) knots 


210 (or 170) knots 

Subject 

Run 2 

Run 4 

Run 6 

Run 8 

Run 10 

Run 12 

Run 14 

Run 16 

1 

MAN, R 17-2 

GM, R 17-3 

DICE, R 17-4 

CSM, R 17-5 

MAN, R 21-2 

DICE, R21-3 

CSM, R 21-4 

GM, R21-5 

2 

MAN, R21-3 

CSM, R 21-4 

DICE, R 21-5 

GM, R 21-6 

MAN, R 17-3 

DICE, R 17-4 

GM, R 17-5 

CSM, R 17-6 

3 

MAN, R21-4 

CSM, R 21-5 

GM, R21-6 

DICE, R 21-7 

MAN, R 17-4 

GM, R 17-5 

CSM, R 17-6 

DICE, R 17-7 

4 

MAN, R 17-5 

DICE, R 17-6 

CSM, R 17-7 

GM, R 17-2 

MAN, R 21-5 

CSM, R21-6 

GM, R21-7 

DICE, R 21-2 

5 

MAN, R 17-6 

GM, R 17-7 

CSM, R 17-2 

DICE, R 17-3 

MAN, R21-6 

DICE, R21-7 

GM, R 21-2 

CSM, R 21-3 

6 

MAN, R 17-7 

DICE, R 17-2 

CSM, R 17-3 

GM, R17-4 

MAN, R 21-7 

GM, R 21-2 

CSM, R21-3 

DICE, R21-4 

7 

MAN, R21-2 

DICE, R21-3 

GM, R21-4 

CSM, R 21-5 

MAN, R 17-2 

CSM, R 17-3 

DICE, R 17-4 

GM, R 17-5 

8 

MAN, R 17-3 

GM, R 17-4 

DICE, R 17-5 

CSM, R 17-6 

MAN, R21-3 

DICE, R 21-4 

GM, R 21-5 

CSM, R21-6 

9 

MAN, R21-4 

CSM, R 21-5 

DICE, R 21-6 

GM, R 21-7 

MAN, R 17-4 

CSM, R 17-5 

DICE, R 17-6 

GM, R 17-7 

10 

MAN, R 17-5 

CSM, R 17-6 

GM, R17-7 

DICE, R 17-2 

MAN, R21-5 

GM, R21-6 

CSM, R21-7 

DICE, R 21-2 

11 

MAN, R 21-6 

DICE, R21-7 

GM, R 21-2 

CSM, R 21-3 

MAN, R 17-6 

GM, R17-7 

DICE, R 17-2 

CSM, R 17-3 

12 

MAN, R 17-7 

GM, R 17-2 

CSM, R 17-3 

DICE, R 17-4 

MAN, R 21-7 

CSM, R21-2 

DICE, R 21-3 

GM, R 21-4 


116 





Table 4. Inter arrival Errors of Display Formats 


To preserve anonymity, the controller subject numbers have been randomized 
and bear no relationship to the original test order appearance; however, 
listed subject numbers are consistent for all report data. 




Aircraft-pair interarrival errors, sec 



MAN 

GM 

DICE 

CSM 

Controller 

Pat tern- speed 









test subject 

procedure, knots 

Mean 

St dev 

Mean 

St dev 

Mean 

St dev 

Mean 

St dev 

1 

170 

3.20 

15.85 

-0.49 

15.18 

0.14 

14.24 

0.10 

19.72 


210 

4.59 

16.64 

0.07 

9.12 

0.17 

6.39 

-0.41 

12.67 

2 

170 

15.13 

19.41 

-0.71 

17.01 

0.27 

15.53 

0.32 

14.07 


210 

4.44 

17.04 

-0.15 

7.75 

-0.40 

6.45 

0.02 

10.60 

3 

170 

-0.65 

15.80 

-0.07 

10.92 

0.51 

15.96 

0.19 

15.69 


210 

-0.04 

11.07 

0.66 

10.39 

-0.06 

7.42 

0.53 

12.99 

4 

170 

7.29 

16.92 

-0.65 

15.26 

0.20 

14.32 

0.05 

14.91 


210 

1.11 
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Figure 3. Example of 210-knot centerline slot marker display format. Magnetic North is up. 
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Figure 4. Example of 210-knot DICE display format. Magnetic North is up. 
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Figure 5. Example of 210-knot graphic marker display format. Magnetic North is up. 




Figure 6. Example of graphic turn marker generated for anticipated overshoot of ILS centerline. Magnetic 
North is up. 
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Figure 10. Sequence of events that arrival aircraft experiences in TIMER concept. 
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Figure 11. Simulation air traffic controller stations, with final controller and coordinator on left and feeder I 

controller on right. 
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Figure 12. Final controller station, with final controller on left and coordinator on right. 
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Figure 14. Nominal traffic flow (indicated by arrows) for Denver Stapleton arrivals in land-runway 26 
configuration. 


132 



Arndt 46 91038 

ILSRWY 26L 

ATIS ARR 125.6 
DEP 124.45 
DENVER APP CON 
NORTH 127.4 288.1 
SOUTH 120.8 363.0 
DENVER TOWER 
1 18.3 1 19 5 257 8 
GND CON 
1219 257 8 
CLNC DEL 
127.6 257.0 


DENVER/STAPIETON INTL (DEN) 
DENVER, COLORADO 



MISSED APPROACH 
Climb to 6500 then climbing 
right turn to 10,000 direct DEN 
VORTAC then vio DEN R 046 
to PLOTS Int/DEN 17 DME and 
hold 

l-DEN 

G2> 

MM 




osi— 5 n« * r 

CATEGORY 

A | B | C | D 

S-ILS 26 L 

5 5 3 3 / 2 4 200 (200- ’6} 

S-LOC 26L 

5 8 00/ 24 467 {500- Vj) 

5800/40 

467 (500-%) 

5800/50 

467 (500-1) 

CIRCLING 

5880-1 547(600-1) 

5880-1 V5 

547 (600- 1 ’6) 

5900-2 

567 (600-2) 

SIDESTEP 
RWY 76R 

5800-1 477(500 1) 

! 

5800-1 Vi 

477 (500-1 ’6) 

5800-2 

477 (500-2) 


ei ev 5333 [ 

TDZ/CL Rwy 35R 
Rwy 8L Idg 6696' 
Rwy 36 Idg 
6500' 



* A 
"*5396± 

HIRL Rwys 8R-26L, 17L-35R, l7R-35l 
and 18-36 


REIl Rwy* 1 8 and 26R yn7F 
MIRL Rwy 8L$6* 5333 


FAF to MAP 5 .5 NM j 

Knots 

60 

90 

120 

1 50 

180 

Min: Sec 

5:30 

r.*o_ 

2:45 

2:12 

1:50 


39 D 46'N-104°53'W 


LS RWY 26L 

Figure 15. ILS Runway 26L approach plate. 


DENVER, COLORADO 
DENVER/STAPIETON INTL (DEN) 


133 





Frequency Frequency Frequency Frequency 


i 


DICE countdown runway interarrival error, sec 





(a) 170-knot pattern- speed procedure. 


Figure 16. Distribution of lumped (simply combined) interarrival time errors o 
formats. 
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(b) 210-knot pattern-speed procedure with speed adjustment on final. 
Figure 16. Concluded. 






Frequency Frequency Frequency Frequency 


DICE countdown runway excess separation, n.mi. 



Graphic marker runway excess separation, n.mi. 



Centerline slot marker runway excess separation, n.mi. 



Manual runway excess separation, n.mi. 


(a) 170-knot pattern-speed procedure. 

Figure 17. Distribution of lumped (simply combined) excess (relative to minimum requirement) separations at 
runway threshold of all subjects for four display formats. 
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(b) 210-knot pattern-speed procedure with speed adjustment on final 

Figure 17. Concluded. 



25 


ZH 95 percent confidence interval 



5 


I I i 

MAN CSM GM 

170 knots 


DICE 


MAN 


I I L 

CSM GM DICE 

210 knots 


Figure 18. Separation precision of display formats for both pattern-speed procedures. 




Figure 19. Delivery precision effect on single runway arrival rate. 
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(b) Plot of mean vectors for all subjects combined. 

Figure 20. Vectors per aircraft in final sector for four display formats and both pattern-speed procedures. 
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(a) Manual format. 

Figure 21. Flow pattern for all runs combined for 170-knot pattern-speed procedure. 









(a) Manual format. 



Figure 22. Flow pattern for all runs combined for 210-knot pattern-speed procedure with speed adjustment on 





(c) DICE countdown format 
Figure 22. Continued. 
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gure 23. Histogram of controller response times for graphic marker format base-to- final turn advisor (170- 
and 210-knot procedures responses combined). 
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Figure 24. Histogram of controller response times for DICE format base-tofinal turn advisor (170- and 210-knot 
procedures responses combined). 
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Figure 26. Histogram of controller response times for DICE format final speed advisor of 210-knot pattern-speed 
procedure. 


Figure 27. Scatter plot of about 6000 eye-scan fixations during single manual format data run using 210-knot 
pattern-speed procedure with speed adjustment on final. 
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Figure 28. Scatter plot of about 6000 eye-scan fixations during single centerline slot marker format data run 
using 210-knot pattern-speed procedure with speed adjustment on final. 
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Figure 30. In-track fixation time for 
speed procedures for all subjects 


several types of display objects for four display formats and both pattern- 
combined. 
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Figure 31. Illustration of zones defined on final controller’s display for lookpoint analysis. Zone 4 is everywhere 
on display not defined by zones 1, 2, and 3. 
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Figure 32, Mean fixation time among three of four defined display zones for four display formats and both 
pattern-speed procedures. 
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^To preserve anonymity, the controller subject numbers have been randomized and bear no relationship to the original test 
order appearance; however, listed subject numbers are consistent for all report data. 


(a) Tabulation of time for all subjects. 


12 T 



11.21 



MAN GM DICE CSM 

210 knots 


(b) Mean time for all subjects combined. 

Figure 33. Time inside FAF for four display formats and both pattern-speed procedures. 
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(a) Tabulation for both pattern-speed procedures. 



(b) 170-knot pattern-speed procedure. 



(c) 210-knot pattern-speed procedure. 


Figure 34. Mean dwell time for several types of display objects for four display formats and all subjects 
combined. 
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Note: Each zone-pair combination contains the sum of cross-check scans of 
order 4 or greater for 3 separate zone pairs; Zl/All = Zl/Zl + Z1/Z2 + Z1/Z3; 
Z2/A11 = Z1/Z2 + Z2/Z2 + Z2/Z3; Z3/A11 = Z1/Z3 + Z2/Z3 + Z3/Z3. 
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(a) Tabulation for all subjects and 170-knot procedure. 



(b) Tabulation for all subjects and 210-knot procedure. 

Figure 35. Cross-check scans, of order four or greater for zone-pair combinations, all subjects combined, and 
four display formats. 
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(c) Mean CCS’s for all subjects combined and both pattern-speed procedures. 

Figure 35. Concluded. 
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a To preserve anonymity, the controller subject numbers have been randomized and bear no relationship to the original test 
order appearance; however, listed subject numbers are consistent for all report data. 


(a) Tabulation of workload for all subjects. 
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(b) Mean workload for all subjects combined. 

Figure 36. Task- Load- Index- assessed workloads for four display formats and both pattern-speed procedures. 
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(a) Tabulation of ratings and weights. 
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(b) Plot of rating and conventional TLX-assesscd workloads. 

Figure 37. Ratings of source of workload factors for four display formats and both pattern-speed procedures. 
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(a) 170-knot procedure. 

Figure 38. Subject controllers’ mean factor rating (height) and weight (width) and the computed overall 
workload using these values for each display format. 
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(b) 2 10- knot procedure. 
Figure 38. Concluded. 
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Figure 39. Distribution of subject controllers’ relative ratings for “workload or effort required to use the format” 
criterion for four display formats and 170-knot pattern-speed procedure. 
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Figure 40. Mean ratings for “workload or effort required to use the format” criterion and Task Load Index 
assessments for four display formats and both pattern-speed procedures. 
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Figure 41. Distribution of subject controllers’ relative ratings for “ease of adapting to or learning to use the 
format” criterion for three FASA formats and 170-knot pattern-speed procedure. 
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Figure 42. Mean ratings for “ease of adapting to or learning to use the format” criterion and mean practice 
times, taken by subject controllers prior to data runs, for three FASA formats and both pattern-speed 
procedures. 
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Figure 43. Distribution of subject controllers’ relative ratings for “amount of help or benefit in spacing traffic 
on final” criterion for three FASA formats and 170-knot pattern-speed procedure. 
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Figure 44. Distribution of subject controllers’ relative ratings for “workload or effort required to use the format” 
criterion for four display formats and 210-knot pattern-speed procedure. 
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Figure 45. Distribution of display-format relative ratings and format means for “workload or effort required 
to use the format” criterion for those five subject controllers with considerably higher TLX workloads for 
their graphic marker format at 210 knots than at 170 knots. 
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Figure 46. Distribution of subject controllers’ relative ratings for “case of adapting to or learning to use the 
format” criterion for three FAS A formats and 210-knot pattern-speed procedure. 
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Figure 47. Distribution of subject controllers’ relative ratings for “amount of help or benefit in spacing traffic 
on final” criterion for three FAS A formats and 210-knot pattern-speed procedure. 
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Figure 48. Distribution of subject controllers’ relativp ra 
criterion for four display formats and both pattern-: 
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Figure 49. Distribution of subject controllers’ for “case of adapting to or learning to use the format” criterion 
for three FASA formats and both pattern-speed procedures. 
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Figure 50. Distribution of subject controllers’ for “amount of help or benefit in spacing traffic on final” criterion 
for three FASA formats and both pattern-speed procedures. 
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Figure 51. Mean ratings for “workload or effort required to use the format” criterion and the Task Load Index 
assessments for both pattern-speed procedures. 
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(a) Five subjects with considerably higher TLX workloads for graphic marker at 210 knots than at 170 knots. 

Figure 52. Distribution of relative ratings for the “workload or effort required to use the format” criterion for 
four display formats and both pattern-speed procedures. 
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Figure 53. Relative ratings for “workload or effort required to use the format,” “ease of adapting to or learning 
to use the format,” and “amount of help or benefit in spacing traffic on final” criteria for three FASA 
formats and both pattern-speed procedures. 
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